Editor’s Note: Retraction of article containing fabricated quotations

We are reinforcing our editorial standards following this incident.

See full article...

 

[Ragashingo]

Just a reminder for everyone, the article in question got posted and then yanked within two hours on a Friday afternoon leading into a three day holiday weekend. Today is said holiday. We shouldn't expect the results of any investigation or editorial policy changes until later this week. Assuming that because you haven't seen affirmative statements yet means none are forthcoming is jumping the gun.

What I'm not clear on is: Will there be more investigation? I think the keep/fire debate is interesting with good points on both sides, but the Ars retraction doesn't really indicate to me that there's ongoing decision making. If anything, the reference to it being an isolated event says to me that they are done with the investigation and punishment phase...

 

[jnv11]

One of the concerns about the old method of leaving up a retracted article that I thought of is that more AI scrapers would get fooled by a false article. Maybe taking the whole thing down might have to become the new norm when an article has to be retracted and the required humans needed to fix the errors are not available.

Benj admitted to being sick with COVID-19, and different people have different reactions to being sick with that, especially because it could be a relatively milder but somewhat severe problem if the person is vaccinated or could be more life-threateningly severe if the person is not vaccinated.

 

[AI_Skeptic]

On the other hand I don't see how a journalist can report on AI without using AI, essetially they have to dogfood it, and the problem here was that due to a slip twixt cup and lip the dogfood ended up getting into our collective bowl as readers.

There's no reason why a journalist reporting on AI have to use AI on an article they are writing. They can use AI for other purposes, or for their personal use, or to generate output that's not going to be used in an article.

 

[Bad Monkey!]

One of the concerns about the old method of leaving up a retracted article that I thought of is that more AI scrapers would get fooled by a false article. Maybe taking the whole thing down might have to become the new norm when an article has to be retracted and the required humans needed to fix the errors are not available.

Benj admitted to being sick with COVID-19, and different people have different reactions to being sick with that, especially because it could be a relatively milder but somewhat severe problem if the person is vaccinated or could be more life-threateningly severe if the person is not vaccinated.

Leaving the old article up when you don't even know what's wrong with it is completely counterproductive, even if you do mark it as "retracted". Someone is going to have to review it and re-edit so that it's compliant with editorial standards, and all the people who can do that are either on vacation and/or under investigation. We can criticize them for not leaving the article up after they've had a chance to revise it and declined to do so.

 

[SubWoofer2]

What I'm not clear on is: Will there be more investigation?

That the investigation conducted over a holiday weekend while an employee was sick, will be declared finished and completed in full on Tuesday morning?

I would be confident as a complete bystander to suggest: perhaps not. Indeed, "unlikely" comes to mind.

 

[adrianovaroli]

I guess you don't know what the word "except" means?

We know how language works. You can't just say "except for the obvious problem, there is no problem".

Well, I mean, you can. You did. But it doesn't work. You're trying to pull a "60% of the times, it works every time."

 

[Carno]

Even with that somewhat wrong definition, the article wasn't AI slop. Everything in the article was written by humans, except the quotes, and as I recall only the quotes in the latter part of the article will AI hallucinations.

Is your name Benj Edwards? There is literally no way you can know any of that for CERTAIN. We know he uses AI to help him write articles, because he’s told everyone that.

You may find it acceptable for your glass of water to have just a few drops of diarrhea in it, but I do not.

 

[markgo]

Newspapers don't delete articles and just stick an editor's note up instead. Ars has gained decades of trust as THE IT/tech information source.

While I agree in principle, in the case of fabricated quotes, you may be legally required to pull them down.

And without the quotes, the story doesn’t have enough to stand.

 

[timby]

That the investigation conducted over a holiday weekend while an employee was sick, will be declared finished and completed in full on Tuesday morning?

I would be confident as a complete bystander to suggest: perhaps not. Indeed, "unlikely" comes to mind.

This is the thing that I feel is getting lost. This was an article posted on a Friday heading into a three-day weekend.

Ken has said many times that Ars enjoys a surprisingly considerable amount of autonomy as a Conde Nast brand, but it is still a brand of Conde Nast, a goddamn gigantic, multinational corporation with intricate and complicated HR processes, and that's on top of Ars being a union shop. This is not a situation as simple as "bad article that shouldn't have been published was published, fire the writer." That may well be the outcome, but it is not the situation and it is not the process of how these things are investigated and resolved.

These things take time.

 

[Alethe]

Whilst I agree that having the article + retraction is the preferred situation from a human PoV, we are also living in a world where that incorrect information is then being indexed and sucked up into LLMs, making the problem worse. So initial damage control is IMO probably the right move - esp. on a holiday weekend. Same thing that should be done with a bad production release - revert and contain the damage as best as possible in the shortest time possible, then do your root cause analysis and mitigations to whatever came out of that.

Right. We are in the age of AI slop and pervasive misinformation at the slightest provocation. In view of this, I can understand the decision to retract and delete the article wholesale.

 

[pjcard]

Just a reminder for everyone, the article in question got posted and then yanked within two hours on a Friday afternoon leading into a three day holiday weekend. Today is said holiday. We shouldn't expect the results of any investigation or editorial policy changes until later this week. Assuming that because you haven't seen affirmative statements yet means none are forthcoming is jumping the gun.

That was actually part of my point: it's important to be seen to be doing the right thing. As you point out, currently the idea that there will be a full and open investigation is left to be taken on faith.

 

[graylshaped]

Maybe it's because we are both Europeans, but this knee-jerk extreme reaction seems to be a bit of an American thing? Like how some of the sentencing in the US penal system often sounds beyond ridiculous.

When evaluating a debacle (because it IS one) like this, history, patterns and all kinds of other context matters a lot. Only the Ars staff can properly judge prior experience with Benji, but if that was a positive experience, then that should count for a lot. In no insignificant part, because of the backlash of a "zero-fault culture", as you specified. Someone who burns his hand badly, will also be much more careful in the future.

The fact that this happened to a writer focused on generative AI, in an article about generative AI makes it very embarrassing. But at the same time it also puts the spotlight on just exactly HOW alluring, pernicious and disrupting the tech is. It's symbolic for the pressure it is currently applying from all sides, where managing to RESIST the temptation despite everything, could even have serious short term negative effects for that person.

A proper and transparent post-mortem is necessary, though.

Why is this an American thing? Americans weren't the ones who dug up Cromwell's corpse so it could be put executed again, nor invented a device to make the removal of heads publicly more efficient, nor staged the Inquisition.

Let's not bring tribalism into this.

 

[Resistance]

Judging from his statement, copy/pasting text out of ChatGPT and into Ars articles seems to be a regular occurrence. Benj seems not at all concerned about the practice itself, just that it resulted in fictitious quotes. Hey Ken, who else on your payroll is pasting text from LLMs into articles? Is it everyone?

I get the downvotes because to many people here, that's probably an absurd question. Thing is, it's not an absurd question to everyone, and many would be well within their rights to ask it. Another question some might ask: do confidential sources have information they provide put into ChatGPT? Part of the way to address these concerns is through decisive action.

 

[SomewhereAroundBarstow]

I can't believe that what actually led me to subscribe was a comments section devolving into people arguing about the precise technical definition of "AI slop." I hope I don't end up regretting this.

 

[Sarty]

That was actually part of my point: it's important to be seen to be doing the right thing. As you point out, currently the idea that there will be a full and open investigation is left to be taken on faith.

Some form of "we will have more to say on this topic once we have completed our investigation, but that will take some time" should have been the last line of Ken Fisher's statement, and I don't think there was a good reason to omit it.

"We're not used to dealing with fuckups of this magnitude, and it just didn't occur to me to be that explicit" is about the best I can do.

 

[Resistance]

No, you're right, and I'm an American saying this: knee-jerk responses are part of our heritage and rights!

Seriously, over 900 comments? I cannot say that I've EVER seen that many for any story. And yeah, I've been here awhile. This is what we've got for entertainment now, evidently. We can't let anything simmer anymore, it's all full-boil pessimism in this country, 24/7/365. Let the damn company figure out what is going on! My bet, and this was said earlier, is that they did not have a good SOP for this type of problem, and pulled the article as a stop-gap measure. I do hope Ars reposts the original unredacted, along with a proper "Story Update" at the top of the article as they've done in the past when something changes that affects how the article reads. But I don't need it THIS FRACKING MINUTE. Get the facts straight first.

I am in a position in tech where my clients are all clamoring for ALL AI ALL THE TIME NOW NOW NOW. I have to be the adult voice in the crowd asking to simply slow down. This technology is being driven so hard right now that I can't even keep up some days! And we all know what happens when we can't keep up... tools like AI get made! But it is not, in no uncertain terms, going away. Feel how you will about that, but that's the honest truth. Power has come to the uninitiated and the overworked, and they like it. It's an "equalizer" for better or worse. How people use it is what it will become. If companies would take a few beats to teach people ways to use the technology in a safe and sane fashion, we'd be better off, but time waits for no robot chat session.

Finally, Benj... damn dude, I understand it was a seemingly innocent slip, but with great power... I do not want to see you thrown to the wolves, but I also need to see something more than 'mea culpa' after this. I want to see the full, unadulterated behind-the-scenes breakdown of what went wrong and where; I'd like that to be written by Benj. I also want to see Ars stand behind their employees and give this person a second chance. I want to see others learn from his mistake(s). I want to see other publishers learn from ARS' mistakes as well!

But mostly? I want us all to learn that no man is perfect, no AI is perfect, let Ars and the community we have here accept this and let the team move on with real reporting. Please.

I got you: 86 thousand and counting.

 

[adrianovaroli]

Except that's not what I said. Please don't lie by putting false words in my mouth. What I actually said was "everything in the article was written by humans, except the quotes".

I'm not mincing words here, it is abundantly clear what the sentence means and if you try to argue some semantic 'gotcha' technicality, it just means you have no substantive argument to speak of. You're free to phrase it however you want; the meaning of the sentence is still the same.

What you wrote:

Everything in the article was written by humans, except the quotes

In your text, as you wrote it, "everything" is an important word. "except the quotes" is a copout, a secondary thing, less important. Your emphasis is on the very large part of the article that was written by humans.

And the problem is, as other people have also said, the fabricated quotes are the important bit here. "I have been a honest person all my life, except for my crimes." "Everything is fine, except for the disaster"

Yeah, the sentence is very clear. We're telling you. It just doesn't mean what you want it to mean. You want to emphasize the "everything" and... It doesn't matter. You can't. Sorry.

 

[BelowTheRoot]

With everything said that could possibly be said, can someone tell me why this story isn’t showing up in the Top 5 under most read? For me, I’ve never visited a story so many times in the last three days.

 

[AI_Skeptic]

That would be bad. I still don't think that's a fair analogy.

To stay with the car analogy: he made a huge scratch on the front door. It's ugly and totally unprofessional. But he then apologized profusely, took full responsibility, and told me he will do whatever he can to fix it.

So, to be clear: nobody got permanently hurt, and he wants to fix it.

As I find it rare for people to own their mistakes, and as there is no pattern of this being a systematic problem, I am leaning heavily towards forgiveness.

You can call me a shitty leader if you like, but I treat my team the same way. The result: they're HIGHLY productive, NEVER afraid to own up to their mistakes, and we have a healthy culture where discussing f*ckups and fixes is never toxic or dangerous. That goes for me to: I am secure enough in my leadership position to share my own f*ckups with them on equal terms. In leadership terms, this is called "psychological safety," and it's the means to make high-performers stick with their job for a very, very long time.

Here's why I (and others) are leaning towards not supporting him or firing him:

1. He is supposed to be the subject matter expert on AI/ LLM. It is clear he used an LLM tool without understanding HOW an LLM tool actually operates (it's a character generator, it doesn't have any intelligence), and therefore any output needs to be verified - which could have been verified easily by taking the LLM output, pressing Control+C to copy it, going to the website, pressing Control+F (for Find), and then pasting the output in the search box to see if there's a match.
2. His claim of "I have COVID-19 so I tried a new tool" rings hallow in my ears. When people are sick with a fever, and need to do something, they revert back to the processes they are most familiar with, since they don't have the ability to learn or integrate new knowledge easily.
3. A LLM-based tool that allows quote extraction COULD be an amazing technology (assuming it's not Python +Regex). An AI expert should make a note of that technology, and file it as "let me investigate it when I feel better. This could make a good article".
4. The explanation he gave is missing some steps. He got an error message using a tool developed in Claude, he went to ChatGPT to figure out the error message, and ChatGPT gave him the quote. I'd like to know what happened in the step between "Ask ChatGPT about the error message" and "ChatGPT pulled the quote", because that's not how ChatGPT works.

 

[pjcard]

Some form of "we will have more to say on this topic once we have completed our investigation, but that will take some time" should have been the last line of Ken Fisher's statement, and I don't think there was a good reason to omit it.

"We're not used to dealing with fuckups of this magnitude, and it just didn't occur to me to be that explicit" is about the best I can do.

I think that's entirely plausible. I sincerely hope too much damage is not done in the meantime.

Edit: corrected spelling

 

[OOPMan]

Which author was it? There were two credited. This is a load of bullshit being served to us.

UPDATE: It was Edwards:
View: https://bsky.app/profile/benjedwards.com/post/3mewgow6ch22p

What a surprise /s

 

[Rudde]

Was this like the situation, last May, that was an embarrassment to the Chicago Sun-Times?

https://www.npr.org/2025/05/20/nx-s1-5405022/fake-summer-reading-list-ai

Here is a better post-mortem: https://chicago.suntimes.com/news/2...n-king-fake-book-list-errors-sun-times-review
I expect a similar post-mortem from Ars.

 

[The Lurker Beneath]

Maybe, maybe not. Depends on the perspective, perhaps outrage, you want to bring.

In that instance the fakery was the guts of the story. The story collapsed without the invented bits.

Whereas in this case, the body text carried the essentials of the story notwithstanding the "quotes".

Ah, but that was an old-style LLM from last summer. Claw AI would have written and self-published all the books in the fake reading list!

 

[ajmas]

Exactly. When newspapers publish corrections they tell you what is being corrected, not just that something was disappeared. The same for research journals.

That is certainly an issue in this retraction article, though at the same time there are newspapers that will publish a correction, but burry it deep in a section nobody is going to notice.

 

[Resistance]

Even with that somewhat wrong definition, the article wasn't AI slop. Everything in the article was written by humans, except the quotes, and as I recall only the quotes in the latter part of the article will AI hallucinations.

If people call it ai slop then it meets the definition of ai slop, that's how definitions work.

 

[LeeF]

Benj admitted to being sick with COVID-19, and different people have different reactions to being sick with that, especially because it could be a relatively milder but somewhat severe problem if the person is vaccinated or could be more life-threateningly severe if the person is not vaccinated.

I've been pretty diligent about getting vaccines for COVID-19. The first time I got infected was a few months after the first round of vaccines. It hit me while I was in the middle of cleaning several years of cruft out of my water heater - I soldiered on for about an hour, then I crashed and had to stop for a bit. I meant to get right back to it because there was a steady drip I couldn't stop, but I just couldn't make myself get up and finish for several hours, and the drip overfilled the bucket I placed to catch it and soaked the whole area around the heater, and did some damage to the drywall. I was vaccinated, I didn't have any respiratory distress at all, just a mild cough, but it just completely took the wind out if my sails.

Ars has been my #1 go-to source for tech news for decades now, and I have a great deal of trust in the editorial staff. This incident was deeply concerning to me, but I have faith that they will make the right call. I kind of wish the pitchfork crowd would exercise a bit of patience, give Ars time to do their thing, and see what the outcome is, before passing judgment.

 

[adrianovaroli]

Writing a complete and accurate sentence is not "a copout", it's how you communicate in English. If you seriously don't understand what the word "except" means, or what the concept of context means, all I can say is go take a course in remedial English. The person I was replying to was suggesting that the entire article was "AI slop", suggesting the entire article was generated by an AI tool. That wasn't the case; as I said already, everything but the quotes was written by the two human authors, per Benj's own commentary on BlueSky.

In no way was I saying there is no issue here, or some dumb folksy shit like "except for the problem, there is no problem" so you can stop trying to attribute that false stuff to me. You are really not making the "got 'em" dunk that you think you are.

The quotes, IIRC, were interspersed throughout the last five paragraphs of the text. I'd say I'm way more correct in thinking you're using "everything except" wrong than you are by saying the entire article wasn't AI slop. You can insist your sentence is accurate all you want, it's still not true. The context is precisely what makes it not true. Sorry.

 

[Resistance]

I mean sure, literally also means figuratively if you want to go down that route. The English language is a living one, but one can't just use blatantly misuse extremely heavily-used words with well-defined meanings in a novel, lying way and then not except to be called out about it.

Sure but that's not what happened here. There is nothing novel or deceptive about calling an article that contains sloppy ai generated content, ai slop.

 

[Quasius]

I'm not highly invested in this specific incident, but it's yet another example of why I despise what the "AI" plague is doing to human discourse.

 

[Resistance]

I disagree, but you do you.

So you don't believe that there is anyone who won't have their trust in ars fully restored as a result of Edwards being put on unpaid leave?

 

[nimelennar]

Except that's not what I said. Please don't lie by putting false words in my mouth. What I actually said was "everything in the article was written by humans, except the quotes".

While I don't want to level any accusations against Benj without evidence, "Everything in the article was written by humans, except the parts where it's possible to prove whether or not it's AI generated by comparing against the source purportedly being quoted" has a blush of naivety to it.

I'm sure it happens that when someone gets caught doing something unethical, that they haven't done anything wrong in addition to what they've been caught out for. And it's entirely possible that that's the case here.

At the same time, it's also a convenient claim for someone who's been caught out to make when it isn't actually the case, that is, when there's further wrongdoing that can't be, or hasn't yet been, proven.

Which, of course, is why a breach of trust is such a big deal in the first place.

 

[Solidstate89]

Biggest fuckup Ars has made since you people ran that 4 part podcast series with noted bigot sam harris.