Editor’s Note: Retraction of article containing fabricated quotations
- Thread starter JournalBot
- Start date
If it's interesting context or if anybody cares regarding "active comments section", that comment section ran for about two pages (definitely sure) in maybe ninety minutes (less sure).
Upvote
27
(27
/
0)
I actually think the original poster was accusing Dr. Mole of already using AI when writing her articles.
Upvote
8
(12
/
-4)
Resistance
Wise, Aged Ars Veteran
I'm pretty sure I've read every comment in this thread (up to #430) and I have evolved my opinion throughout.
For the purposes of argument (and because it won't change the conclusion (except where otherwise noted)) I will take the most charitable interpretation of what has happened and that interpretation is:
When working as an author of any kind, you must own your output, you are responsible for what you produce, that requires you to take the upmost care in reviewing anything you did not write yourself, including quotes, references, and output of computer tools including but not limited to LLMs.
I personally do not believe that the above interpretation rises to the the level of requiring a termination (this is that otherwise noted part) however there is more that needs to be considered:
Even if I may not think he should go, the risks that come from retaining him are significant enough that I lean on the termination side of things. If there were no new developments beyond an update stating what happened I would continue to read ars and would not be dissuaded from subscribing, but I'm not everyone. The opinion of the community and the risks to the culture of the team are high enough that the best outcome would probably be a resignation, after that, a termination.
For the purposes of argument (and because it won't change the conclusion (except where otherwise noted)) I will take the most charitable interpretation of what has happened and that interpretation is:
When working as an author of any kind, you must own your output, you are responsible for what you produce, that requires you to take the upmost care in reviewing anything you did not write yourself, including quotes, references, and output of computer tools including but not limited to LLMs.
I personally do not believe that the above interpretation rises to the the level of requiring a termination (this is that otherwise noted part) however there is more that needs to be considered:
Even if I may not think he should go, the risks that come from retaining him are significant enough that I lean on the termination side of things. If there were no new developments beyond an update stating what happened I would continue to read ars and would not be dissuaded from subscribing, but I'm not everyone. The opinion of the community and the risks to the culture of the team are high enough that the best outcome would probably be a resignation, after that, a termination.
Upvote
29
(32
/
-3)
This has happened before on Ars and every time it happens it is incredibly disappointing. FFS Ars wont let you delete or edit comments after a grace period, but the much more important front page reporting can get memory holed or edited to be a totally different article, with no way to read the original article.
Upvote
64
(66
/
-2)
I don't think "deleted"; the mods/admins have expressed an unwillingness to do that except for first-post trolls. Hidden/moved seems a better bet, especially given the "You do not have permission to view this page" error when you try to access the thread; when something is deleted from the forum and you try to access it, you typically get a Moonshark, not a permission error.
Upvote
20
(22
/
-2)
That may have been a result of some closer over site of her work. Do you not remember the two articles she wrote early on (regarding NFT's if I remember correctly) that got torn to shreds in the comments similar to this one?
Upvote
-19
(1
/
-20)
I wouldn't say no to Jim returning but I'd prefer he focus on ZFS for my own selfish reasons.
Upvote
21
(21
/
0)
Benj, AI reporter, trusting AI and getting burnt by a hallucination reminds me of Eric, space reporter, recent non-critical transcription of Elon's "xAI datacentres in space" lunacy - in that Ars could maybe use some introspection to make sure y'all are holding feet to the fire over here, and not allowing your clear passion for these topics to cloud you into drinking too much Kool-aid?
Upvote
116
(120
/
-4)
I don't think 5a happened before 6. Listing it first implies Ars tried to shut down the conversation while leaving the article up, and I don't know that it actually happened that way. Of course once the original article was pulled, it makes sense that the comments would also be hidden, that has happened before.
Upvote
21
(21
/
0)
The sad reality is that the "AI" bullshit generator industry is undermining any kind of media. On YouTube when I step out of my bookmarked and largely trusted channels I feel inundated with completely made up or at least significantly slop-contaminated videos to a degree that the platform is becoming unusable for discovering anything new.
Which also makes it more and more important to have quotes and sources independently double-checked by separate humans which apparently has not been done here, or the garbage would have been detected before publication as it should have been!
Upvote
39
(40
/
-1)
I've been reading Ars for over 20 years. Kyle and Benji are long time contributors. I don't know what the correct administrative actions are, but I trust Ken and co. to do the right thing. Let's not get hasty with the pitchforks for two long time contributors, just because they made a mistake. (Admittedly a very big mistake, but we all make mistakes.)
Upvote
16
(38
/
-22)
The comments were locked somewhere around 3:30 Central (quoting from my own memory here), and the article remained up for, eh, a few tens of minutes thereafter. But the duration was short enough that it could well have been the time it takes between clicking one button and clicking another.
Upvote
24
(24
/
0)
That's just how LLMs work. They produce text that is a high-likelihood possible follow-up to the text that is in the prompt. There's no "thinking" about meaning. If you prompt an LLM for an article, and articles typically have quotes, then the LLM will produce an article with quotes, because that's the kind of text that it is emulating. The likelihood that the quotes will be real is not particularly high because that's irrelevant to how LLMs work.
This applies to all sorts of things. For example, I have played with having LLMs rewrite or otherwise improve my resume for a particular job, and consistently what they do is add skills or experience that a person likely to get the job would have. Are these additions correct? No, of course not, it's 100% lies.
Upvote
75
(75
/
0)
What are you even talking about? Sure that this paragraph wasn't auto-generated without any comprehension?
That's what a thoroughly vetted autopsy would and should be for!
Upvote
-13
(2
/
-15)
Upvote
2
(3
/
-1)
Whatever happens to the authors is Ars, and maybe Conde Nast's business.
That said, having two authors let past such a egregious thing is pretty disappointing. Multiple eyes on the text is supposed to be a gate for catching stuff like this.
And all that said, failure is only a disappointment if something isn't learned from the experience. And this one is a target-rich environment, from journalists to other practitioners to the AI shepherds themselves. I'll be happy to see Ars write an article on the whole thing, including their role.
That said, having two authors let past such a egregious thing is pretty disappointing. Multiple eyes on the text is supposed to be a gate for catching stuff like this.
And all that said, failure is only a disappointment if something isn't learned from the experience. And this one is a target-rich environment, from journalists to other practitioners to the AI shepherds themselves. I'll be happy to see Ars write an article on the whole thing, including their role.
Upvote
19
(21
/
-2)
Resistance
Wise, Aged Ars Veteran
i want to lodge a formal complaint with the administrator of this server
this mmo kind of sucks right now, this whole LLM plotline is dumb bullshit
the mechanic shouldn't even be part of the game
this game was broken enough before LLMs were introduced
it feels like this game jumped the shark when the gorilla was shot
*Yes I'm aware of the other much more significant issues like random permanent debuffs applied during character creation and economy balance issues, I'm pantomiming an entitled whale.
this mmo kind of sucks right now, this whole LLM plotline is dumb bullshit
the mechanic shouldn't even be part of the game
this game was broken enough before LLMs were introduced
it feels like this game jumped the shark when the gorilla was shot
*Yes I'm aware of the other much more significant issues like random permanent debuffs applied during character creation and economy balance issues, I'm pantomiming an entitled whale.
Upvote
36
(47
/
-11)
Yes, it was two pages (63 comments). I actually have the first page saved, and would post a link to an archive of it if I didn't think Ars might get cranky about it and ban me.
Upvote
21
(21
/
0)
A question for Ars:
Do you hire fact checkers?
Aside from the still-unanswered question of how this fake text made it into the article in the first place, there’s also the question of how it made it through the editorial process.
Do you hire fact checkers?
Aside from the still-unanswered question of how this fake text made it into the article in the first place, there’s also the question of how it made it through the editorial process.
Upvote
41
(43
/
-2)
Have edited (again) to try to make it a bit clearer. I don't think the exact order of that event is particularly material, but have tried my best anyway. There were probably several actions wrt comment threads, and again I don't think the order is exactly important:
- locking the original comments thread
- changing the article to a retraction, or at least, changing it to something different in the interim
- creating a new comments thread with the single comment
- unlinking the original thread from the article
- moving the original thread to a hidden forum
- linking the new thread to the article
Upvote
1
(1
/
0)
I had actually read Scott Shambaugh's blog post around 3 a.m.Friday because one member posted it in the comments of the OpenAI article. When I saw the ars article about it, I naturally wanted to learn more and when I saw Scott's comment, I went and scanned his blog post and confirmed his blog post hadn't changed, (because some speculated he may have changed the post, etc).
So, I saw the whole thing go down and the article disappear, along with the locked comment threads in the forum and so forth.
However, my initial reaction was completely opposite from yours because I felt that trying to immediately memory hole it and stop all discussion didn't reflect well on their journalistic integrity.
All readers deserve to have that information available in order to understand the issues being discussed in this article. We literally had people commenting here asking what they're referring to or having to be internet sleuths to reconstruct the order of events and issues.
They needed to have kept the article up with a bold disclaimer at the top stating that the accuracy of the content is under review, why, and that they would be posting the results of their investigation as soon as possible.
They can still recover by putting the article back up and citing the specific errors, irregularities, how those came to be in the article, and what they are going to do to prevent future incidents.
Upvote
57
(57
/
0)
The honorable thing would be for Benj to resign without waiting to be fired. Ars should publish the resignation letter as part of the post-mortem process. I can't believe it took 12 pages of comments before someone says this. 9 pages, it seems.
I realize there are real ramifications for unemployment benefits and health insurance when you resign instead of being fired, but it is still the right thing to do.
As for the article itself, clearly you don't want the AI hallucinations on the web because they will spread. That said, the rest of the article is fundamentally sound. It should be put back up with a heading note explaining what happened, and most importantly, with the AI garbage removed. Mark the deletions. Maybe put in legit quotes in their place. That should fix the issues.
Put the comments back. Ars placing a comment at the place where the original comments ended.
And yes, a full and public post-mortem.
I also wonder how much influence the parent company Conde Nast is exerting here.
I apologize for any typos because I am seeing serious input lag while typing this.
Edit: Ninjad on the resignation:
Right here, three pages ago.
Thank you, Resistance, for pointing that out.
I realize there are real ramifications for unemployment benefits and health insurance when you resign instead of being fired, but it is still the right thing to do.
As for the article itself, clearly you don't want the AI hallucinations on the web because they will spread. That said, the rest of the article is fundamentally sound. It should be put back up with a heading note explaining what happened, and most importantly, with the AI garbage removed. Mark the deletions. Maybe put in legit quotes in their place. That should fix the issues.
Put the comments back. Ars placing a comment at the place where the original comments ended.
And yes, a full and public post-mortem.
I also wonder how much influence the parent company Conde Nast is exerting here.
I apologize for any typos because I am seeing serious input lag while typing this.
Edit: Ninjad on the resignation:
Right here, three pages ago.
Thank you, Resistance, for pointing that out.
Last edited:
Upvote
-14
(14
/
-28)
Resistance
Wise, Aged Ars Veteran
I hadn't considered this, reduced credibility with sources could be one of the biggest negative consequences for ars should Edwards continue to be employed. And it might not manifest in particularly obvious ways, fewer high quality tips, more guarded responses to requests for comment, more refusals to comment, less collaboration in general.
As much as I want Edwards to stay, that outcome would be so toxic to ars that the negative consequences would likely vastly outweigh the positive ones.
Upvote
40
(41
/
-1)
Resistance
Wise, Aged Ars Veteran
Right here, three pages ago.
As for resignation vs firing. If Edwards is going to leave ars and he has sufficient savings to make benefits not too big an issue, I think resignation makes future career prospects as a journalist better than if he was fired.
Upvote
26
(26
/
0)
I fired my share of people over the years. I also did NOT fire my share of people over the years for what at face value would have appeared to be career-ending conduct. It's a process. Gather the facts. Sleep on it. Consult. Then, whichever way you go, own it.
What one doesn't want to do is make an emotional, knee jerk decision before taking a step back.
Upvote
111
(111
/
0)
I've been a member for a long long time, and while I've been almost entirely in lurk mode for the last few years, I've gladly paid my yearly membership fee because of the community, and the long form articles that come out every so often that have made ArsT a resource well worth the cost.
Watching this episode unfold though, was a real, real, real disappointment. In this era of rapid 'enshitification' and the infectious spread of AI in to everything around us, I always want to find, and support content that relies on a human process to make sure that the content its producing, is accurate, and - you know, actually - produced by humans ...
The flood of AI Slop which is making it hard to know whats real, and whats not, for a lot of people on this planet, makes the human element, a human process, and journalistic standards -- that much more important.
My membership actually renews, on Tuesday (in two days), and I'm going to let it, but there's an opportunity here for ArsT to step up its game - and build a practice that supports human production of content (even if you use AI for assistance in the basic things) - with a human editorial process, that sets its apart from the rest of the garbage on the web. That maintains your legacy of being a quality source for accurate and well researched news.
I hope this happens.
Watching this episode unfold though, was a real, real, real disappointment. In this era of rapid 'enshitification' and the infectious spread of AI in to everything around us, I always want to find, and support content that relies on a human process to make sure that the content its producing, is accurate, and - you know, actually - produced by humans ...
The flood of AI Slop which is making it hard to know whats real, and whats not, for a lot of people on this planet, makes the human element, a human process, and journalistic standards -- that much more important.
My membership actually renews, on Tuesday (in two days), and I'm going to let it, but there's an opportunity here for ArsT to step up its game - and build a practice that supports human production of content (even if you use AI for assistance in the basic things) - with a human editorial process, that sets its apart from the rest of the garbage on the web. That maintains your legacy of being a quality source for accurate and well researched news.
I hope this happens.
Upvote
43
(45
/
-2)
I have been a regular reader of Ars for over a decade. While Benj's linked apology is definitely worthwhile (a good apology makes a huge difference vs. a defensive-asshole stance), I believe any journalistic or creative enterprise should take a zero-tolerance stance towards AI tools. My initial instinct is that the writer making the error should be fired.
But I just said a good apology makes a difference, and I believe that. I would like to see a full postmortem published next week, with Benj's apology included in full, and a statement that he (and all Ars staff) will refrain from using AI tools in their work going forward, full stop. With appropriate apologies and lesson-learning, I think I am fine with a one-strike rule. Otherwise, Ars is losing a reader.
But I just said a good apology makes a difference, and I believe that. I would like to see a full postmortem published next week, with Benj's apology included in full, and a statement that he (and all Ars staff) will refrain from using AI tools in their work going forward, full stop. With appropriate apologies and lesson-learning, I think I am fine with a one-strike rule. Otherwise, Ars is losing a reader.
Upvote
21
(25
/
-4)
I am surprised that the authors don't seem to have read the room about the perception of AI by Ars readers. Like, read the comments of the articles these guys have published in the past and they are not very supportive. The articles where they review and test new LLM models; what do they do first? Ask it to tell a bunch of dad jokes. Get real. Readers (at least those that leave comments) on this site seem to have very little tolerance for confabulations and slop text generated by the chatboxes. You would think the authors might have noticed that by now and avoided using them at any point in their writing process.
Most of the authors on this site have such a good writing voice that I recognize them without reading the by-line. Two sentences into a Wired article posted here I say yuck, realize its a Wired article, and then click the back button. The comment from @Jim Salter a few pages back made a really good point. Ars is supposed to be one of the best tech sites on the internet. And in my opinion that is because the authors are knowledgeable and put out quality pieces. I originally subscribed to Ars because of Siracusa's OS X reviews. Now it's down to only about 2-3 authors that I look forward to reading. Neither author was in that top 3. Kyle's gaming coverage was always good, but now I realize he was also the one doing the dad joke AI reviews.
Most of the authors on this site have such a good writing voice that I recognize them without reading the by-line. Two sentences into a Wired article posted here I say yuck, realize its a Wired article, and then click the back button. The comment from @Jim Salter a few pages back made a really good point. Ars is supposed to be one of the best tech sites on the internet. And in my opinion that is because the authors are knowledgeable and put out quality pieces. I originally subscribed to Ars because of Siracusa's OS X reviews. Now it's down to only about 2-3 authors that I look forward to reading. Neither author was in that top 3. Kyle's gaming coverage was always good, but now I realize he was also the one doing the dad joke AI reviews.
Upvote
64
(64
/
0)
Resistance
Wise, Aged Ars Veteran
I'm on the fence about zero tolerance for using AI in this line of work. First, it's completely unenforceable for remote work. Second, we don't know what the future holds for this tech and its perception. Third, I think the best policy is to have those that submit work be accountable for what they submit, regardless of how it is produced.
Upvote
38
(45
/
-7)
We were led to believe those quotes were taken directly from the blog post, so yes they could have checked by reading the source.
Upvote
76
(76
/
0)
The forest you missed while educating us on the terminology is that there is a whole lot of young smoke surrounding he who is currently the zero-th in line.
Upvote
-1
(3
/
-4)
- Status