Editor’s Note: Retraction of article containing fabricated quotations
- Thread starter JournalBot
- Start date
fellow human
Ars Praefectus
Maybe this is karma for selling out our comments to OpenAI then disabling comment deletion to take away our only tool to avoid it.
Our words helped train the AI that produced the slop in the retracted article. Benji's not the only one with blood on his hands; thanks to Ars we all do.
Our words helped train the AI that produced the slop in the retracted article. Benji's not the only one with blood on his hands; thanks to Ars we all do.
Upvote
-15
(21
/
-36)
Upvote
30
(31
/
-1)
The editorial & review process is still my biggest curiosity about Ars in general.
I wouldn't say it's "frequently" but a couple times a year we get those its-Friday-and-a-new-writer-puts-out-something-clearly-not-up-to-Ars-expectations articles. Those articles get immediately roasted in the comments (rightfully so), then eventually an editor comes in to apologize and do damage control.
What exactly is the review process here? If there is oversight, why can't they predict these articles which crumble after 2 minutes of analysis by the readership? Who has the authority to push the button on posting a new story? How are accuracy and facts being confirmed?
Upvote
71
(72
/
-1)
Resistance
Wise, Aged Ars Veteran
Upvote
64
(64
/
0)
I suggest that using AI to help write the article was unfortunate. Falsifying a quote is unacceptable. Not fact-checking the quotation is unacceptable. Lying about what someone said is a terrible journalistic violation.
Upvote
41
(41
/
0)
I struggle to understand all the people who keep arguing that keeping the article online could be a negative because it could cause other LLMs to have access to bad information. The internet is FULL of wrong information. No person should ever let their actions be dictated by how it might effect the giant moneysink bullshit generators that are current LLMs. I would go so far as to say it might even be NOBLE to poison LLMs. If Sam Altman and his ilk cared about ensuring accuracy, maybe they shouldn't train their products on a torrent (no pun intended) of unfiltered and unverified content scraped en masse.
Either way, whatever action is to be taken should be judged on the merits it has for HUMANS, not products. Does the article staying up with proper context as a monument to a mistake serve the readership and the website better than the article being deleted? Is the subject of the article materially harmed by keeping the article up, or do they also oppose deletion? (hint hint, Ars should be asking Scott Shambaugh their opinion on this). Those are the questions to be asked. The fact that so many people even let concern for LLM 'quality' come to mind as relevant is quite frankly terrifying.
Either way, whatever action is to be taken should be judged on the merits it has for HUMANS, not products. Does the article staying up with proper context as a monument to a mistake serve the readership and the website better than the article being deleted? Is the subject of the article materially harmed by keeping the article up, or do they also oppose deletion? (hint hint, Ars should be asking Scott Shambaugh their opinion on this). Those are the questions to be asked. The fact that so many people even let concern for LLM 'quality' come to mind as relevant is quite frankly terrifying.
Upvote
75
(75
/
0)
Using "they" to refer to an institution like the CJR in this context is perfectly natural English. Is English your first language? It would explain why you seem to be reading a whole different article than others.
Edit: I meant to quote him but messed it up, but it's a few posts up
Edit: I meant to quote him but messed it up, but it's a few posts up
Last edited:
Upvote
40
(40
/
0)
I won't be dropping my subscription, but it's important to note that things aren't actually 'verified' unless that process is done by a human.
Using AI tools is impressive and all, but handing off our work to an LLM is doing nothing but increasing our chances of making a mistake. If you have to verify the LLM's work anyway, it's probably more efficient to do the task yourself. In this case it was reading and cutting and pasting quotes from a blog...
I would not trust an AI to report vital signs readings from a patient in a medical setting, and I sure as hell would double check the readings myself if I did. The stakes, professionally and ethically, are just too damn high. That's not even considering the consequences for the patient if the AI hallucinated.
I assume Journalism has the same standards.
Using AI tools is impressive and all, but handing off our work to an LLM is doing nothing but increasing our chances of making a mistake. If you have to verify the LLM's work anyway, it's probably more efficient to do the task yourself. In this case it was reading and cutting and pasting quotes from a blog...
I would not trust an AI to report vital signs readings from a patient in a medical setting, and I sure as hell would double check the readings myself if I did. The stakes, professionally and ethically, are just too damn high. That's not even considering the consequences for the patient if the AI hallucinated.
I assume Journalism has the same standards.
Upvote
48
(48
/
0)
I generally agree with your assessment, I would just reiterate my recommendation that ars should indeed properly research, verify and then publish a postmortem analysis capping off this incident, if possible also a related but separate article illustrating the greater issue as it concerns publishing in general now (since this would be a very different type of article it should remain separate but linked).
This cannot be quick if done properly and we should not push for a quick-but-sloppy report, but it should be done.
Upvote
27
(27
/
0)
Whataboutism completely unrelated to the breach of ethics at hand.
Upvote
22
(27
/
-5)
I had respect for your position right until you said you were going to read the articles you no longer trusted anyway.
Upvote
46
(51
/
-5)
This really deserves a “I used AI uncritically and this is how I screwed up” follow-up article to accompany all of the ones about lawyers messing up their careers with AI citations.
Add me to the list of people who feel that the retraction was not handled at all well and the original article should have been updated.
Add me to the list of people who feel that the retraction was not handled at all well and the original article should have been updated.
Upvote
58
(58
/
0)
I've found AI to be terrible as a search engine too, and a leading cause of the degradation of Google's search results in recent years.
Upvote
53
(53
/
0)
I would just say the response is incomplete at this point, but the completion is just not possible before the legal fallout has been processed and before the evidence is fully analyzed.
Upvote
38
(38
/
0)
I can't wade through 15 pages of comments on this, I just wanted to post as a subscriber, and reader of just over 20 years, this has seriously damaged and degraded my opinion of Ars. I am wavering on cancelling my sub as I want to believe there are still great writers here well worthy of support (Beth, in particular, I think is an important voice in calling out the garbage going on in the health field in U.S. currently). I understand we all err, but hell you all cover this stuff and are ostensibly subject matter experts, I just can't fathom the same publication that featured multiple stories on A I. hallucinations and fabrications, like lawyers getting caught for false and fabricated case history and etc... turns around and does it to themselves?? This is just one opinion and token bit of feedback from a long time fan but geez, gang. It feels like a betrayal.
Upvote
69
(73
/
-4)
I was a journalist at the beginning of my career, and I was credibly threatened with a lawsuit over a misquoting - not a fabrication, just a mistake - that necessitated the physical retraction of all undistributed printed copies, followed by my resignation and the resignation of my editor.
Those losses were the most instructive experiences I had as a journalist. They didn't end my career as a journalist, nor as a writer; they strengthened it.
With the utmost compassion, Benj should not be writing for Ars going forward. Full stop.
Those losses were the most instructive experiences I had as a journalist. They didn't end my career as a journalist, nor as a writer; they strengthened it.
With the utmost compassion, Benj should not be writing for Ars going forward. Full stop.
Upvote
81
(82
/
-1)
SubWoofer2
Ars Tribunus Militum
For mine, google's full-force enshittification dates to 2019 and its strategy change to being an advertising company, no longer a search company.
SEO had been causing grief for some years prior.
The big major hint to future enshittification was dropping phrase search, about 2011 IIRC. No more [item]AND[item].
None of these is AI.
Upvote
42
(42
/
0)
Hypothetically, you could use an LLM to create a software tool that downloads the content of a webpage, identifies the quote marks, and copies the text between them. That would not be subject to hallucination. You could also create a tool that asks the LLM to identify the quotes, then use a text search to verify a match.
Or you could create a tool that asks the LLM for the quote itself, that would be problematic. Or you could capture the quote as above but ask the LLM to identify the speaker of the quote, in which case the quote text could be reliable and the attribution iffy.
We already know the tool didn't work at all in this case, so it's a bit of a tangent.
Upvote
7
(14
/
-7)
I don't mean to argue with your post, but to use it as a good clear statement to respond to: I think that constitutes malice, or something very near to it. If the Ars policy forbids using AI tools for an article, why were AI tools being run on article[-adjacent] text? What was he "testing", if the tool could never be used for his work product by policy already?
Even the "innocent" explanation to me seems on its face to be an admission of AI tool use against Ars policy. If it wasn't, then I think the policy needs to be: No AI use from start to end, at any point in the process, period.
Upvote
50
(50
/
0)
The only thing that's hypothetical about that is using an LLM to do it. I could write this in either JavaScript or PHP using RegEx to match patterns with a very high degree of accuracy and my degrees are in Communications, not Computer Science.
Upvote
40
(40
/
0)
We've known the explanation for years, now: we so strongly associate natural language manipulation with real human actors, that the illusion is very very powerful, and immediately bamboozles you the moment you let your guard down. Especially because it doesn't give away any clues, and has no internal pressure of any kind (fear, self-preservation, reputation...) to "not fail ypu".
Upvote
41
(41
/
0)
It will be interesting to see how all of this plays out with regards to how LLMs will answer queries about this in the future...
Upvote
4
(4
/
0)
My subscription hinges a lot more on ars' response to the writer's malfeasance!
Although this certainly speaks to stricter article verification standards being necessary.
Upvote
16
(20
/
-4)
There's often a distinction made between using AI to analyze, understand and check content, as opposed to using it to generate content.
For example, some software companies allow AI coding agents to be used for assisting in finding bugs, searching complex code-bases, explaining what code does, and checking for sub-optimal implementations... but not for actually generating production code.
However, this incident highlights what is already becoming apparent in the software space. If someone is given a tool that is capable of generating content, and that tool confidently and persuasively presents the content to the user, then it's human nature to grab it and use it (often without checking that it is even valid).
Here, Benji said he was simply using AI to help explain and analyze blog posts. In the process, it spat out a series quotes, and he just decided use them. Maybe he went in with the best intentions, but like so many others, he's fallen into the trap of blindly trusting the outputs.
AI use is a slippery slope. Users intend to just get some "writing prompts", or to "explain some source material", or to use it to "help determine which algorithm is best". Then before they know it, despite going in with the best of intentions, they're copying and pasting dubious content into their work.
Upvote
30
(36
/
-6)
Generative AI is unbeatable at "producing content". In a world run according to today's incentives, that is a grave curse.
I'll keep repeating it: it is - in many different ways - a universal acid that will keep eating through foundations until morale improves.
Upvote
38
(38
/
0)
Snarky Robot
Ars Legatus Legionis
It has been a long time since I commented on Ars. While it wasn’t the primary reason I stopped subscribing and commenting, one part of it was that I had some issues with what I felt was declining trust in reporting, specifically with Eric Berger burying a story that implicated both Elon Musk and the CEO of SpaceX in some sordid drama involving a younger, very junior employee in a short, incomplete blurb in the rocket report, rather than in actually covering it as a story, the way that many other publications handled it.
I was also disappointed here. Once again, it felt like a person whose job is to cover a subject allowed themselves to get themselves ensnared in it. If you’re going to cover AI, and you use generative slop tools to assist in your work, yeah, it’s going to make people question how much you let your personal benefits color your work, or at least how much you’re going to let negative feedback on your subject affect you. And while nothing Benj has written is as bad as the doers/checkers debacle, there is definitely reason for people who are more skeptical of generative slop to question whether Ars’ coverage was in line with what they were expecting.
This whole situation sucks. I was absolutely livid that Ars deleted the original article without comment. I have always thought about retractions as something that should should be public and in-line. Having read replies here, I’ve relented on that a bit. There is something to be said about how, in the age of AI scraping, removing it can mitigate the risk of it being scraped and furthering the harm. That’s especially true because Condé Nast sells their stuff to slop factories. I’m not sure the best path on this, but I will say I think disappearing the article should at least come with a new article explaining it [Edit: that article, even if very short, should probably be contemporaneous]. Even if you turn off comments on the new article immediately, it at least creates a public record and doesn’t leave people thinking a situation is being ignored or that the focus is on PR, not fixing the problem. Deleting it may be better than in-line editing in the modern world. But 2 days of silence on it didn’t do Ars any favors.
As for the author…ugh. My first instinct like many others, is straight-up that termination is in order. This doesn’t appear to be straight-up malicious making up of quotes. But the credibility of the institution took a big hit over it, even if the offense is sloppiness and not an intentional deception. Journalism is IMPORTANT. Good journalism is damned near SACRED in the world we live in. So to an extent, getting rid of someone who is willing to take shortcuts is normal. But…well, journalism is in a shitty place and running people who make mistakes out the profession may hurt it more by discouraging people from putting themselves in that position.
I think my biggest takeaway is…I think Ars’ writers are unionized and I think that’s good. My personal vindictive streak is very much in the camp of this being a fireable offense. But I don’t always love my vindictive side, so it’s good that, if nothing else, this is a decision that takes a little time and is one where the author has protection. I hope Ars keeps this process semi-transparent.
My second takeaway is that I think it’s good that Ars addressed this sooner than Tuesday. Obviously, it’s the weekend, so getting all the stuff sorted in the background and making these calls ate into that. Aurich coming in and doing some cleanup of the comments is also very welcome. I think finding the line on this subject likely requires a deft touch in letting people vent but not, you know…vent their whole ass spleen to the point it becomes about their personal politics. So far, that touch has been present, which is great.
I was also disappointed here. Once again, it felt like a person whose job is to cover a subject allowed themselves to get themselves ensnared in it. If you’re going to cover AI, and you use generative slop tools to assist in your work, yeah, it’s going to make people question how much you let your personal benefits color your work, or at least how much you’re going to let negative feedback on your subject affect you. And while nothing Benj has written is as bad as the doers/checkers debacle, there is definitely reason for people who are more skeptical of generative slop to question whether Ars’ coverage was in line with what they were expecting.
This whole situation sucks. I was absolutely livid that Ars deleted the original article without comment. I have always thought about retractions as something that should should be public and in-line. Having read replies here, I’ve relented on that a bit. There is something to be said about how, in the age of AI scraping, removing it can mitigate the risk of it being scraped and furthering the harm. That’s especially true because Condé Nast sells their stuff to slop factories. I’m not sure the best path on this, but I will say I think disappearing the article should at least come with a new article explaining it [Edit: that article, even if very short, should probably be contemporaneous]. Even if you turn off comments on the new article immediately, it at least creates a public record and doesn’t leave people thinking a situation is being ignored or that the focus is on PR, not fixing the problem. Deleting it may be better than in-line editing in the modern world. But 2 days of silence on it didn’t do Ars any favors.
As for the author…ugh. My first instinct like many others, is straight-up that termination is in order. This doesn’t appear to be straight-up malicious making up of quotes. But the credibility of the institution took a big hit over it, even if the offense is sloppiness and not an intentional deception. Journalism is IMPORTANT. Good journalism is damned near SACRED in the world we live in. So to an extent, getting rid of someone who is willing to take shortcuts is normal. But…well, journalism is in a shitty place and running people who make mistakes out the profession may hurt it more by discouraging people from putting themselves in that position.
I think my biggest takeaway is…I think Ars’ writers are unionized and I think that’s good. My personal vindictive streak is very much in the camp of this being a fireable offense. But I don’t always love my vindictive side, so it’s good that, if nothing else, this is a decision that takes a little time and is one where the author has protection. I hope Ars keeps this process semi-transparent.
My second takeaway is that I think it’s good that Ars addressed this sooner than Tuesday. Obviously, it’s the weekend, so getting all the stuff sorted in the background and making these calls ate into that. Aurich coming in and doing some cleanup of the comments is also very welcome. I think finding the line on this subject likely requires a deft touch in letting people vent but not, you know…vent their whole ass spleen to the point it becomes about their personal politics. So far, that touch has been present, which is great.
Upvote
93
(95
/
-2)
anguisette
Wise, Aged Ars Veteran
eh, that's not exactly what they said. they just said they don't want to financially support Ars if it uses LLMs, which could be for any number of reasons, like a moral objection to LLMs.
i don't subscribe to Ars, because i didn't like their previous handling of situations like this. that doesn't mean i think everything Ars publishes is full of lies, or that i can't generally trust things written here, even if there are specific writers i will never read because i them them untrustworthy. i did stop reading for a while though, and came back recently... so this incident doesn't have great timing as far as trust-in-the-editorial-process goes and i'll be keeping a close eye on what happens here.
but in general, i think it's fine to say that you find something useful enough to read but distasteful enough, for whatever reason, that you don't want to explicitly support it financially. i read The Guardian with an adblocker, because i mostly trust them to provide factually accurate information, but there's zero chance i'll ever give money to help that bigoted rag push their anti-trans editorial propaganda.
Upvote
32
(37
/
-5)
anguisette
Wise, Aged Ars Veteran
we can all agree, hopefully, that whatever happens to the author(s) involved as a result of this, it's a good thing they will have the union on their side to ensure they're properly represented and aren't being scapegoated to hide any issues that might exist in the broader editorial process.
Upvote
33
(34
/
-1)
This is really where we're at. Humans associate polished communication with intelligence. A system that can produce text that sounds sophisticated and reads well is deemed "intelligent".
This human tendency has frequently led to problems. There are plenty of "gurus" out there who spout nonsense, have a muddled worldview, but speak with such authority, confidence and fluidity that they attract droves of followers. Meanwhile, there are scientific geniuses out there who can't string two words together and struggle to get their ideas across, even to their peers.
LLMs are like the gurus. Full of empty words, but articulate. You can ask ChatGPT to argue a given point of view, and it will create a very persuasive argument. You can then ask it to argue the opposite position and it will be equally convincing. It's not intelligent. It has no insight or opinion on these matters. It's just a great tool for stringing text together (using a fundamentally lossy representation of information to generate that convincing text).
The sci-fi of my youth got it wrong. We were meant to have "AI" that had infallible, always-accurate knowledge, but struggled to articulate it in anything but a robotic monotone. Instead, we get "AI" with lossy and inaccurate knowledge, but which expresses it with such articulation that we trust it anyway.
Upvote
60
(60
/
0)
Thank you for this post.
I hadn't even considered the potential for other AI agents using the original article as a source, then effectively republishing the hallucinated "quotes" and further amplifying them. Perhaps removing the original is a better solution given that potential, even though it's wildly unconventional to do so.
I look forward the oribital HQ's follow ups.
Upvote
-2
(9
/
-11)
They had enough time to blackhole the original article, replace (by recycling the URL) the content wholesale with a retraction notice but left the original text out as well as removing all the original comments. They disabled the comments section on the
It's important to see what was retracted. What the problem was that warranted the withdrawal.
The "easy middle of the night oh shit" reaction stopped being forgivable when they didn't un-unpublish the original, comments and all, and put the mea culpa on it at the top, complete with original by-line.
Oh, and it looks like they're banning people that commented on other articles from Benj asking if that article contained fabrications too (I can't see a reason why this person was able to post yesterday, posted 5 times total in 7 years, and got banned within the day. Sure, it's easy off-topic warning territory, maybe even delete-worthy or other moderation steps, but banning? Seems a little much.)
Upvote
22
(35
/
-13)
- Status