Editor’s Note: Retraction of article containing fabricated quotations

Status
Not open for further replies.
To each his own. Looking at the facts of the case, I really don't understand where that "hell no" comes from.

The victim admits that, while the quotes were not his, the meaning was entirely something he might have said. As such, the fake quotes were not damaging. They are a theoretical issue more than anything.

The overall content of the article, while containing hallucinated quotes, was correct and faithful to the core substance.

The author has explained how the error occurred, and that shows a severe lapse in judgement but not a systemic issue. In other words zero malice and zero intent to push AI slop to get a paycheck.

These are the facts. Did you read his account?

I'd be much more inclined to trust him never tk repeat that mistake than any other author out there.

Yes I would share articles by that author. And perhaps especially so if he and Ars made an actual article about all this, so a much wonder audience can learn the dangers.

The entire world is high on LLM-hype. This could have happened to any media (journalists are not less probe to lapses in judgement than lawyers, developers, civil servants, etc.).

Learning and moving on is the clear better choice.
I don’t wish bad on Benj and truly hope this isn’t career ending, but I can’t trust his writing. Especially his writing about AI. I actually started out thinking I hope he doesn’t lose his job, but the comments that you incorrectly characterize as pitch forks actually raised good points.

You need to think about ARS and the other writers. The paying subscribers you’re telling to calm down are the people paying their money. If they keep Benj on, and enough paying subscribers leave, that lessens the income they need to pay their writers. Telling upset people to calm down isn’t going to keep them around. Ad income would also go down from free readers who lost trust and left.

It is unacceptable for a senior AI reporter to use AI for any part of articles except any clearly marked examples of LLM output. LLMs are well known to screw things up. I can understand how lazy lawyers would trust AI. They’re falling for the AI bullshit because they aren’t educated on the subject. I think they’re stupid, but I can understand it.

I had been considering subscribing lately, but I will not if AI slop is allowed. I’ve already stopped reading anything written by Berger, and I recall an incident where another author was rudely telling someone in the comments asking questions to google it.

The only thing I look forward to on here anymore is Beth’s horror stories. I don’t want to lose yet another good news website because I can’t trust the article to get the most basic things correct. There’s already so few left that I’m starting to miss web rings.
 
Upvote
41 (51 / -10)

graylshaped

Ars Legatus Legionis
67,695
Subscriptor++
They disappeared the article with no acknowledgement or explanation.

Three days later, they have acknowledged it but provided little context or information on what actually happened, and left it up to the author of the piece to explain it on his personal Bluesky account.

That is not how retractions work in "most other news media sources."
Nope. Retractions usually appear in a tiny box days or weeks later, if at all. More reputable sources will give the retraction "comparable" placement, as in, the tiny box will be on the lower fold of the front page if the mistake was a headline article. Sometimes an edit--not a retraction, but an edit--will be made to an article where the error is corrected and the change noted with an "ed:" note somewhere in the article, not always at the top.

That second approach really wasn't an option here without the active engagement of the author's article, who for more than one reason was not in a position to help identify the errant quotes. "Pull it and we'll figure it out" was the choice made.
 
Upvote
18 (19 / -1)

KobayashiSaru

Ars Praefectus
4,178
Subscriptor++
I'll pile on with my 2c. Regardless of whys and what happened. This was handled in a mediocre way.

Weird thing to say when they haven't stated they are finished handling or won't take more action after they are finished investigating it after the holiday weekend.
 
Upvote
-7 (17 / -24)
Post content hidden for low score. Show…
Sorry how dare I try to provide a reasoned perspective instead of screaming into the void like all the people who created accounts just to complain about this.:rolleyes:
Ah, was that what you were going for when you accused other people of performative outrage? That was your reasoned perspective? Interesting.
 
Upvote
28 (38 / -10)

taxmanmike

Seniorius Lurkius
14
Subscriptor
I'm looking forward to the RCA/postmortem. In the interim I'm hoping to see an additional note added to this article to indicate the RCA is in progress. If that happens I'll be reserving further judgement assuming the results surface in a reasonable time; probably by EOW. If there's no RCA/postmortem (or it isn't shared) I'll be disappointed.
 
Upvote
17 (18 / -1)

JoHBE

Ars Praefectus
4,134
Subscriptor++
@Ken Fisher, ignore that noise above.

This was a serious error. A major cockup. But you acknowledged it. You’ve likely addressed it internally, reinforced your standards, and moved on. It's a serious matter - but it’s NOT a pattern. And without a pattern, termination is absolutely not justified.

Let me be clear: I take the completely opposite view. I will consider cancelling my subscription if you don’t stand by your staff in a situation like this. Teach them accountability. Show us transparency. Do NOT fire them!

I’m a leader myself, so I’ll say this plainly for everyone in this thread: a one-time mistake is not a firing offense for employees who otherwise perform well. Not even when its a very big mistake. If you enforce a zero-fault culture, you don’t get higher standards ... you get silence, blame-shifting, and people hiding mistakes instead of fixing them.

Accountability and transparency goes out the window.

When commenters calls for termination they are, frankly, disproportionate and short-sighted. Shame on you all! It’s easy to "demand consequences" when you don’t carry the responsibility of managing people. In reality, leadership means balancing quality of work with fairness - and recognizing that even competent people sometimes make mistakes. There is always a reason, and sometimes it's a DAMN GOOD ONE!

The right response isn’t to fire someone. It’s to be transparent, fix the issue, and learn from it. In fact, this could be a great opportunity. Explain what went wrong. Show how you corrected it. Let readers see the process. In short: write a whole article about it. Let the flawed human himself write it. Then publish it. That builds credibility far more than a stupid symbolic firing ever will.

If the person otherwise performs well, firing them is just damn wrong. The world needs MORE humans - flawed or otherwise - not LESS. And whoever is responsible for this is now incredibly wiser, stronger, more experienced. Don't throw them under the bus because the stupid court of public opinion calls for it. It's also the Trump-thing to do. So don't do that!
Maybe it's because we are both Europeans, but this knee-jerk extreme reaction seems to be a bit of an American thing? Like how some of the sentencing in the US penal system often sounds beyond ridiculous.

When evaluating a debacle (because it IS one) like this, history, patterns and all kinds of other context matters a lot. Only the Ars staff can properly judge prior experience with Benji, but if that was a positive experience, then that should count for a lot. In no insignificant part, because of the backlash of a "zero-fault culture", as you specified. Someone who burns his hand badly, will also be much more careful in the future.

The fact that this happened to a writer focused on generative AI, in an article about generative AI makes it very embarrassing. But at the same time it also puts the spotlight on just exactly HOW alluring, pernicious and disrupting the tech is. It's symbolic for the pressure it is currently applying from all sides, where managing to RESIST the temptation despite everything, could even have serious short term negative effects for that person.

A proper and transparent post-mortem is necessary, though.
 
Upvote
20 (33 / -13)
Post content hidden for low score. Show…
I don't really want to say how the responsible party/parties should be treated. It's not my place.

On the other hand, if you find yourself commenting
The victim admits that, while the quotes were not his, the meaning was entirely something he might have said. As such, the fake quotes were not damaging. They are a theoretical issue more than anything.

The overall content of the article, while containing hallucinated quotes, was correct and faithful to the core substance.

The author has explained how the error occurred, and that shows a severe lapse in judgement but not a systemic issue. In other words zero malice and zero intent to push AI slop to get a paycheck.
I would suggest you take a good long look at yourself and consider what ever made you type that because it's appalling. Oh so the quotes this time were correct and faithful, that excuses the massive journalistic fault.

To be very clear: while I don't speak for the other 24 pages of commenters, this is not about the honor of mr. Shambaugh. It's about journalistic standards. You don't forge quotes. You don't.

And if you push an article out while you're feverish and use AI tools that inadvertently add slop, and you get paid... there's more than zero intent to push AI slop to get a paycheck.
 
Upvote
49 (60 / -11)

Sarty

Ars Tribunus Angusticlavius
7,816
Sorry how dare I try to provide a reasoned perspective instead of screaming into the void like all the people who created accounts just to complain about this.:rolleyes:
Contrary to the usual semi-conspiratorial viewpoint on paid actors or bots, creating an account and leaving your first comment is actually rather high-friction. If somebody has never gone through the effort to create a user account and leave a comment but they specifically chose to do so for this story, the baseline expectation is that they actually felt very strongly about this, far above and beyond their reaction to any normal story they read on Ars Technica. Discounting their first comment viewpoint, purely on the basis of it being their first comment, is probably the opposite of what you should be doing.
 
Upvote
62 (64 / -2)
Post content hidden for low score. Show…
Post content hidden for low score. Show…

Sarty

Ars Tribunus Angusticlavius
7,816
You'll only see this if you manically refresh the page, but what you can't see is that "KobayashiSaru" has made two different edits to comment #934, each more spittle-flecked than the last.

I don't know what kind of life experience drives such strident defense of fraudulent work behavior. It's foreign to me.
 
Upvote
11 (22 / -11)

TROPtastic

Ars Tribunus Militum
1,676
Subscriptor
You did - this post has the ejection notice and Aurich mentioned it in this post as well. There may be some irony of getting ejected for modifying a quote in this comment thread but we'll need Alanis to chime in to be sure.
Hopefully Benj Edwards gets "ejected" from writing AI articles for a bit. It would be absurd to hold us commentators (not least of which the legendary Jim Salter) to a higher standard than paid writers.
 
Upvote
32 (41 / -9)
Post content hidden for low score. Show…

El.Mich.

Smack-Fu Master, in training
25
Dear Mr. Fisher!

Thanks a lot for clarifying on this delicate subject! And to really simply admit to a failure or an error "takes balls". :cool:

Though looking and reading from good old Germany you reminded me once again why I started back in the late nineties and never really stopped reading ars.

Thank you once again and pls. do keep up the good work! It's urgently needed, especially in this mess of the world!
I obviously was a bit too quick with my comment above! Unfortunately I cannot change it anymore but after having read lots of other comments here and digging a bit more into the depths of this case I side with all the other readers and commenters that think that just taking an article back is definitely not enough! At least in this case!

To prevent similar cases to happen at all the original article should be re-instated and clearly marked as how serious and proper journalism is NOT to be done!

Further steps and measures are probably also to be taken and installed but at least admitting this completely wrong behaviour is the necessary though not sufficient first step ... More are to follow!

Credibility and trust take a really long time to build but they can easily be destroyed completely in just seconds with merely one wrong action taken.

And afterwards takes even much much longer to rebuild them!

If this rebuilding can seriously ever be done!

PS
Take for example anything made out of glass or china / porcelain that gets broken:
Very often, if you can really find all the little pieces, you can try to glue them back together and get something like the original function / some kind of "good working state" back.

But the traces of the glue and the original breaking thereof will always be visible!

How much these traces do influence you in your daily behaviour depends on a lot of other things ... But something really seriously broken can sometimes be glued back together to a working state but not without visible traces of the original sin!
 
Last edited:
Upvote
0 (4 / -4)

Sarty

Ars Tribunus Angusticlavius
7,816
Hopefully Benj Edwards gets "ejected" from writing AI articles for a bit. It would be absurd to hold us commentators (not least of which the legendary Jim Salter) to a higher standard than paid writers.
This combination of factors was also not lost on me. The magnitude of that potential contradiction gives me an additional degree of confidence that the final resolution of this saga will be a rather severe one.
 
Upvote
20 (21 / -1)
The policy was already pretty clear, and not fabricating quotes seems pretty obvious (AI or not), so this is not exactly helping as a statement. One head should have rolled. Two different writers were listed on the byline; which one is it so I can avoid them in the future? I would assume Benj at this point.

UPDATE: this was indeed Benj (blaming it on COVID?). I personally am not interested in seeing him fired, but I switched to BazQux for my RSS feeds for that very reason; it allows me to filter out articles by keyword. So "Benj Edwards", enjoy the company of "Trump" and "Musk"...
Judging from his statement, copy/pasting text out of ChatGPT and into Ars articles seems to be a regular occurrence. Benj seems not at all concerned about the practice itself, just that it resulted in fictitious quotes. Hey Ken, who else on your payroll is pasting text from LLMs into articles? Is it everyone?
 
Upvote
22 (35 / -13)

pjcard

Seniorius Lurkius
36
I haven't posted in a long time. I used to have a subscription to ars but stopped it when I took a break from looking at news websites for my health. I wanted to provide that context for why I feel I still have some small right to have an opinion on this.

I am surprised people are giving ars (or conde nast really) such credit for the retraction: the person who was misquoted essentially gave them no choice. What concerns me is what they have not done.

They have not admitted which story it was.
They have not admitted who the authors were.
They have not explained how it happened.
They have not explained how they will prevent it happening again.

And most concerningly to me they have not disabled advertising on the article. So they are profiting directly from this retraction, just as they did on the original story. That may well be a mistake or a technical limitation, but it brings into question the motivation of the retraction.

The authors responsible, I am lead to believe, are experienced long-term journalists. That makes the need for absolute candour all the more important, and it's absence all the more concerning.

I have a lot of time and indeed love for ars. This saddens me greatly, and leaves me to question an awful lot.
 
Upvote
39 (48 / -9)

danrien

Wise, Aged Ars Veteran
172
Subscriptor
Oftentimes "I was sick" is a legitimate excuse. In this case, it appears to be a partial excuse, one which I accept. The other part of the explanation is a confession of poor judgment, which I also accept. If this was truly a one-off, then it's disgusting and reprehensible for people to be condemning Benj (or Ars) without knowing for certain that this reflects a pattern.

At this time, we can raise an eyebrow and say, "You fucked up, Benj." But if this was a one-time occurrence, then people would do well to recall all the times they fucked up and ask themselves if they would approve of their consignment to hell by all the perfect people who never made a serious mistake.

If there is a pattern of this kind of bad judgment, then, yeah, then have at it and judge away.
Did he say anything to indicate that he isn't using AI tools to write his articles when he's healthy? It only became apparent in this article because the person he misquoted had to say something in the article comments. Everything he has written and writes in the future should now be assumed to be inaccurate first, and needing verification, and that burden shouldn't be put on me as a reader. That's why I pay journalists to investigate newsworthy items and accurately report them to me. Sorry if he is sick, but something he was clearly already doing when healthy, and he just "oopsied" his double checking while sick, completely broke my trust not just in him, but in Ars Technica as a whole (frankly this doesn't spare Kyle either, if he knew benj was sick then he should have also paid extra attention to what he was doing).
 
Upvote
42 (46 / -4)
No editor is going to go check quotations on a benign sentence. It’s entirely on the author to be truthful, and write their own work.

So I'm not a journalist nor do I have any knowledge as to how sites like Ars work but that's not true.

In a more extreme example, when I was a graduate assistant, one semester I was a part-time cite checker for academic publications in my field. My job was to literally check every single citation to make sure that it actually said what the manuscript it was in asserted that it did. And the easy part of this was when the thing the manuscript said was a direct quote, then all I had to do was to make sure that the quote was exact, not taken out of context and that the entire relevant text, not just the part quoted, was being accurately represented in the manuscript. The hard part (in the case of my field like 99% of the the citations) was to verify that the source technically supported the thing it was being used for.

And then a couple of years ago I was quoted in a mainstream publication and well before the article ran I received a call from one of the publication's fact checkers to confirm that the quote and paraphrasing the article was going to use were all accurate.

Anyway, the TLDR here is that although I have no idea what the journalistic standards for such things are nor which publications do it or to what extent, fact checking like this is done.
 
Last edited:
Upvote
63 (65 / -2)

Resistance

Wise, Aged Ars Veteran
418
Neither one of this questions is unanswered, FYI. We know the answer to both:

1. The fake text made it into the article because Benj told an AI tool to present salient parts of the article verbatim to him and it did not do that despite advertising that it could; being Benj' first use of the tool, he (wrongly) trusted it to do as it advertised and did not verify the quotes.
2. The quotes in question were not particularly substantive to the article, nor did they impact the character of the subject/author of said 'quotes'. That part, on top of the bed of historic trust/body of work from Benj in the past, led the co-author and/or editor (I don't know how many people reviewed the work before publication) to not go back and verify the quotes for authenticity.

I guess you are not familiar with the concept of unpaid leave? It's a real, painful consequence which takes up most of the huge amount of space between "do nothing" and "fire him".
Unpaid leave will not reliably remediate the damage. Edit: to expand, it will remediate some of the damage, but for many that action will be insufficient to restore trust.
 
Last edited:
Upvote
1 (9 / -8)

SubWoofer2

Ars Tribunus Militum
2,550
However, if sources are to be verified by an editor, then the kind of editor normally tasked with such verification would be a copyeditor. (And most copyeditors I know are already overworked.)

Ah, yes, the copy editor. Haven't heard that phrase in years, perhaps decades. No- one's willing to pay for what is perceived as a QC role.

The publishing equivalent of having someone else check your code.
 
Upvote
30 (31 / -1)

Ragashingo

Ars Tribunus Militum
2,666
Subscriptor
To each his own. Looking at the facts of the case, I really don't understand where that "hell no" comes from.

The victim admits that, while the quotes were not his, the meaning was entirely something he might have said. As such, the fake quotes were not damaging. They are a theoretical issue more than anything.

The overall content of the article, while containing hallucinated quotes, was correct and faithful to the core substance.

The author has explained how the error occurred, and that shows a severe lapse in judgement but not a systemic issue. In other words zero malice and zero intent to push AI slop to get a paycheck.

These are the facts. Did you read his account?

I'd be much more inclined to trust him never tk repeat that mistake than any other author out there.

Yes I would share articles by that author. And perhaps especially so if he and Ars made an actual article about all this, so a much wonder audience can learn the dangers.

The entire world is high on LLM-hype. This could have happened to any media (journalists are not less probe to lapses in judgement than lawyers, developers, civil servants, etc.).

Learning and moving on is the clear better choice.

Correct and faithful to the core substance does not cut it. Quotes are quotes. For a reason. I totally get there's a valid side that says keep him around and let him learn from the mistake, because one would expect he might be more careful in the future like you said, but wallpapering over journalistic integrity and practice is not right. Keep or fire, we need to demand better journalism and not accept such sloppy article.
 
Upvote
58 (58 / 0)

Readercathead

Ars Tribunus Militum
1,712
Subscriptor
My last comment about it got downvoted to hell for some reason, but I’m not letting it go. I still want to know exactly why Kyle got these fake quotes, worked on the final article and it was still a redundant poorly written mess. “I was so sick that I tripped on the carpet and my drafts went through some AI tools” is a weird explanation, but how did a set of perfectly healthy eyes not find the time or wherewithal to proofread and/or edit the parts of the that weren’t quotes at all? And why was he instructed not to comment?
Kyle’s byline was on that disappeared story but Benj has admitted on social media to be the one at fault for using the hallucination machines. However, as Kaibelf states correctly Kyle also has responsibility for these errors. What was his role in allowing this slop to get published on the Ars website? Presumably he was brought in to help a sick colleague, was he completely unable to do basic fact checking? Don’t they both have an editor and a manager who would also fact check? Why didn’t anyone call the person being written about instead of scraping stuff off the internet like some YouTuber.
 
Upvote
5 (13 / -8)

torque2k

Ars Praetorian
495
Subscriptor++
Maybe it's because we are both Europeans, but this knee-jerk extreme reaction seems to be a bit of an American thing? Like how some of the sentencing in the US penal system often sounds beyond ridiculous.
No, you're right, and I'm an American saying this: knee-jerk responses are part of our heritage and rights!

Seriously, over 900 comments? I cannot say that I've EVER seen that many for any story. And yeah, I've been here awhile. This is what we've got for entertainment now, evidently. We can't let anything simmer anymore, it's all full-boil pessimism in this country, 24/7/365. Let the damn company figure out what is going on! My bet, and this was said earlier, is that they did not have a good SOP for this type of problem, and pulled the article as a stop-gap measure. I do hope Ars reposts the original unredacted, along with a proper "Story Update" at the top of the article as they've done in the past when something changes that affects how the article reads. But I don't need it THIS FRACKING MINUTE. Get the facts straight first.

I am in a position in tech where my clients are all clamoring for ALL AI ALL THE TIME NOW NOW NOW. I have to be the adult voice in the crowd asking to simply slow down. This technology is being driven so hard right now that I can't even keep up some days! And we all know what happens when we can't keep up... tools like AI get made! But it is not, in no uncertain terms, going away. Feel how you will about that, but that's the honest truth. Power has come to the uninitiated and the overworked, and they like it. It's an "equalizer" for better or worse. How people use it is what it will become. If companies would take a few beats to teach people ways to use the technology in a safe and sane fashion, we'd be better off, but time waits for no robot chat session.

Finally, Benj... damn dude, I understand it was a seemingly innocent slip, but with great power... I do not want to see you thrown to the wolves, but I also need to see something more than 'mea culpa' after this. I want to see the full, unadulterated behind-the-scenes breakdown of what went wrong and where; I'd like that to be written by Benj. I also want to see Ars stand behind their employees and give this person a second chance. I want to see others learn from his mistake(s). I want to see other publishers learn from ARS' mistakes as well!

But mostly? I want us all to learn that no man is perfect, no AI is perfect, let Ars and the community we have here accept this and let the team move on with real reporting. Please.
 
Upvote
1 (30 / -29)

Wheels Of Confusion

Ars Legatus Legionis
75,398
Subscriptor
Credibility takes a really long time to build but it can be completely destroyed in just seconds with merely one wrong action taken.

And afterwards takes even much much longer to rebuild it!

If this rebuilding can seriously ever be done!
Sorry for reposting this in images, it's difficult to find the original in a way that people can get to ethically. It's directly about the slow boil of enshittification but it applies beyond pricing incentives. The point is that a company has to be constantly earning its readership's loyalty and trust, they can't take it for granted and assume there will be timely warning signs to change course and maintain their readership like they used to.

1771273434820.png

1771273446410.png

1771273462483.png

1771273471793.png
 
Upvote
75 (82 / -7)
I have learned to never trust either writer (unless one of them wants to step forward and take responsibility). Now I have 3 writers that my subscription pays for and yet receive no value from (the 3rd writer covers pop-culture drivel). Best Ars can do is replace 1 (or both) writers with someone trustworthy.
Benj took sole responsibility.
 
Upvote
17 (17 / 0)
Post content hidden for low score. Show…
Post content hidden for low score. Show…

SubWoofer2

Ars Tribunus Militum
2,550
Upvote
20 (20 / 0)
I haven't posted in a long time. I used to have a subscription to ars but stopped it when I took a break from looking at news websites for my health. I wanted to provide that context for why I feel I still have some small right to have an opinion on this.

I am surprised people are giving ars (or conde nast really) such credit for the retraction: the person who was misquoted essentially gave them no choice. What concerns me is what they have not done.

They have not admitted which story it was.
They have not admitted who the authors were.
They have not explained how it happened.
They have not explained how they will prevent it happening again.

And most concerningly to me they have not disabled advertising on the article. So they are profiting directly from this retraction, just as they did on the original story. That may well be a mistake or a technical limitation, but it brings into question the motivation of the retraction.

The authors responsible, I am lead to believe, are experienced long-term journalists. That makes the need for absolute candour all the more important, and it's absence all the more concerning.

I have a lot of time and indeed love for ars. This saddens me greatly, and leaves me to question an awful lot.

Just a reminder for everyone, the article in question got posted and then yanked within two hours on a Friday afternoon leading into a three day holiday weekend. Today is said holiday. We shouldn't expect the results of any investigation or editorial policy changes until later this week. Assuming that because you haven't seen affirmative statements yet means none are forthcoming is jumping the gun.
 
Upvote
57 (59 / -2)
Post content hidden for low score. Show…
Status
Not open for further replies.