Editor’s Note: Retraction of article containing fabricated quotations

We are reinforcing our editorial standards following this incident.

See full article...

 

[Zlotnick]

Edit: I'm halfway though, it's a good read and is absolutely something people in this thread would be interested in.
Edit 3: Seriously, read it, it's basically a bunch of seemingly highly qualified individuals giving their takes on AI use in journalism with lots of citations and examples (I've got like 10 new tabs open and I'm not finished.

From the article:

I take solace in two historical parallels rooted in craft. When the camera was invented, painters feared for their jobs. But instead Impressionism emerged—a new form of painting not replicable through cameras. Photography liberated painters from realism, allowing a new focus on the subjective experience using visible brushstrokes and techniques that cameras couldn’t capture.

Ah yes, Impressionists -- famous for totally not starving. No parallels between "freeing" Impressionists from the drudgery of paying gigs and people's concerns about AI.

Also, how is letting Impressionists plumb the depths of human experience in any way the same as AI slop? What good is free time in a late stage capitalist hellscape where we race to the bottom or sleep on the street?

 

[Buchliebhaber]

Well this has blown up quite a bit more than I expected after my first early post here.

I've read through quite a bit of this thread to try to glean what happened. It seems pretty clear that Benj screwed up in a way that's pretty bad from a journalistic perspective. I'm pretty convinced it was an honest mistake, but I'm also quite a bleeding heart - I've had to fire people before, and it's never a pleasant process even when it's justified. I wouldn't wish that on others for doing a brain fart when they were sick.

I mean, he's a guy whose job is to write about AI on the regular, as he was assigned to do, so naturally he uses the tools to be able to review them, and even uses the tools in his own workflow to better be able to understand them so he can write better articles about the subject. Presumably, up until now he's been diligent about ensuring that he's not nabbed by AI hallucinations, and missed this time due to job pressures and being sick.

All of which leads to some concerning things.

1) Why didnt the review process catch the quotations that didnt actually exist? The whole point of a review process is to verify this exact sort of thing. There's defintely some major issues with the review process that did not catch this before publication, especially knowing that the author is someone who uses AI tools in their workflow for purposes of learning. It seems that extra scrutiny should have been the norm for Benj's articles given his assignment, and knowing how tech-savvy the Ars readership is and presumably knowing how negatively it would react to just this sort of thing happening.

2) Why was Benj feeling the need to rush to publish this article while sick? Is there too much pressure on the Ars authors? Is there too much pressure on the Ars editors/reviewers that also led to this miss, causing what is undoubtedly significant reputational harm to Ars? This happening at all makes me wonder what the Ars work culture is like - we dont tend to get too much insight into it from the outside looking in naturally, so has it turned into a typical US corporate culture, trying to extract more blood from too few stones? I'm just trying to figure out in my head why Benj felt the need to publish this while sick - did a manager/editor not tell him "you're sick, take a few days off and get better"? Did no one consider doing extra review of his work due to his being sick? This failure suggests an issue in the culture - though it's not a unique one in the US, sadly.

I really like Ars. There's literally no replacement for it. It's what's kept me coming here since 1999 when it was a very young news site. This event is causing a number of people to cancel their subscriptions to send a message or whatever their reason. It has me worried about the future of Ars.

Again, we're outside looking in, and have no transparency into how this happened. I dont expect a public root cause analysis and in-depth report from Ars, though if one was published I would certainly read it with interest. However, something definitely needs to be done, and soon, to address this reputational hit and hopefully start the site on the road to regaining trust among those that have lost it. It's a holiday weekend in the US, so, yeah, I dont envy those working at Ars at all right now, and feel sympathy to them. I've been "in charge" when the shit unexpectedly hit the fan on an otherwise innocuous and quiet holiday weekend, and it really sucks.

However, I hope that this issue is addressed, thoroughly, so that Ars doesn't die. I will be very sad if that happens.

Take care Ars folks - I, like others who have chimed in, still support you guys, and at this point have no intention of cancelling my subscription.

 

[AdrianS]

<snip>
The victim admits that, while the quotes were not his, the meaning was entirely something he might have said. As such, the fake quotes were not damaging. They are a theoretical issue more than anything.

This is an interesting take - the quotes were made up bullshit, but were "truthy enough" that it doesn't matter to you that they were fabricated.

 

[murraytgrub]

This situation just got a mention on my local NPR station.

 

[Wheels Of Confusion]

Some form of "we will have more to say on this topic once we have completed our investigation, but that will take some time" should have been the last line of Ken Fisher's statement, and I don't think there was a good reason to omit it.

"We're not used to dealing with fuckups of this magnitude, and it just didn't occur to me to be that explicit" is about the best I can do.

I think that's entirely plausible. I sincerely hope too much damage is not done in the meantime.

It's not plausible to me. This is not their first rodeo. This isn't even unprecedented in the last 5 years.

I've been loathe to dig up examples because I don't want to be accused of rehashing old points of contention and some of the topics are really sore spots, but this isn't the first article they've nuked (comment thread and all) and their non-response burned a bunch of goodwill and caused a small exodus of readers, subscribers, and community regulars. Ken and Aurich have had to step in then, too. I was one of those calling for a public accounting of the process in the aftermath and certain changes to the way Ars handles things as a matter of course, and we never got it. The changes that were made (which likely directly resulted in Aurich being able to lock things down and get an emergency editorial action so quickly this time) didn't extend to public disclosure about what went wrong and what measures were going to be taken to prevent the issue from happening again.

The "vibrant" discussion you're reading now might even be tame in comparison to what went on just a few years ago when that shit went down. But for many in the community, including dozens I'm still in touch with after they left, the wound was left to fester. Compounding the original mistakes was a sense that the incident was being papered over in hopes that it would just die quietly in the neglect and everyone would just move past it. Well, a lot of people did: they move right off of Ars because of how it was handled.

So I'm not going to cut slack if the whole thing ends here. I've been there before and it's awful. Worse is the implication that after last time, management didn't change its approach. It would be tragic if the same thing happened again when it doesn't have to. That would tell me everything the leadership of Ars really thinks about its readership, let alone the active community. That's why I'm hoping the powers that be at Ars have learned from before and will make different choices that are more responsive to the needs of their readers this time; I have to say that some recent smaller dustups have not left me 100% confident. There's a lot on the line as I see it. At least, there's a lot on the line for me.

Sure but that's not what happened here. There is nothing novel or deceptive about calling an article that contains sloppy ai generated content, ai slop.

This. If a YouTube video contains a bunch of DALL-E graphics, it's slop even if an actual human is in the video doing narration or something.
This became an issue with dubbed animation too. People didn't like the AI-generated audio. The audio was slop even if the video elements attached to it weren't. Therefore the product was slop.

Every time I search for technical information, only to find a mess of auto-generated, referral-link infested slop promoted via SEO, I keep thinking of how a modern web directory could help - even if it only covered a small proportion of sites.

Ultimately, when looking for information, the best option is to go straight to a website that is likely to have the information to begin with. Astronomy gear? Skip the horrendously-incorrect slop sites full of random referral links and go straight to Cloudy Nights or Astrobin. Computer gear? Skip the search engine and look to see if Ars or TechPowerup has a review. Search is rapidly becoming worthless, and we need to fall back on trusting individual sites.

But how to know which sites even cover a topic to begin with? It used to be that you just went to Yahoo Directory or DMOZ to find websites on a particular topic. These days, you need to stumble across them. Returning to a directory-based approach could help with discovery.

But there would need to be some way to keep the slop factories out. If the directory was fully open and they could just spam their multitude of identikit sites in to every category, that would kill the directory as quickly as they killed web search.

Even better would be some kind of carefully-curated list ranked by reputation (as assessed by experts in the field). But getting experts to even agree in the first place would be a challenge.

Something like this is sort-of tenable with the "indie web" of small personal websites that still exist, e.g. Neocities.
Sadly the human affiliate content mill crap made it almost impossible on the Internet at large even before press-button slop generators were available.
These days I'm finding myself doubting general information pages that show up in specific searches unless they predate 2023 or thereabouts. Everything after that I treat as "probably slop." Like, if I see a cool plant growing by the road and want to know if I can find a spot for it on my property? A search might turn up hundreds of pages from the last couple of years on exactly that topic, even that specific plant... which itself seems suspicious.

 

[SomewhereAroundBarstow]

Every time I search for technical information, only to find a mess of auto-generated, referral-link infested slop promoted via SEO, I keep thinking of how a modern web directory could help - even if it only covered a small proportion of sites.

Ultimately, when looking for information, the best option is to go straight to a website that is likely to have the information to begin with. Astronomy gear? Skip the horrendously-incorrect slop sites full of random referral links and go straight to Cloudy Nights or Astrobin. Computer gear? Skip the search engine and look to see if Ars or TechPowerup has a review. Search is rapidly becoming worthless, and we need to fall back on trusting individual sites.

But how to know which sites even cover a topic to begin with? It used to be that you just went to Yahoo Directory or DMOZ to find websites on a particular topic. These days, you need to stumble across them. Returning to a directory-based approach could help with discovery.

But there would need to be some way to keep the slop factories out. If the directory was fully open and they could just spam their multitude of identikit sites in to every category, that would kill the directory as quickly as they killed web search.

Even better would be some kind of carefully-curated list ranked by reputation (as assessed by experts in the field). But getting experts to even agree in the first place would be a challenge.

It's an interesting thought, isn't it? Now I'm remembering a semi-serious/semi-joking paper I wrote in 2008 for an algorithmically generated evaluation system for determining the trustworthiness of social media accounts. I'm not a CS person but I knew there wasn't a good way to do it then and I mostly was trying to show original thinking for the class I was taking. Now I'm like, "Hmm. Something like that could probably be done with AI (or 'AI') to evaluate, categorize, and assign trustworthiness scores to sites in a web directory."

Start working on it now and be ready in the wing when tulip AI mania finishes running its course...

 

[dragoninnaMINI]

Hey guys. I appreciate your rapid response to this situation and the retraction is the right thing to do. However, you're not following optimal journalistic practices here. The article and its associated comments should not have been taken down (and should be restored). The article should've been prepended with a notification that it was being investigated, and again with the retraction once that was decided upon. Failing to leave the text of the article and the comments up is lacking transparency and not what I'd hope to see from a publication that I regard as highly as Ars.

In addition, Ars' LLM/AI policy needs to be further clarified. LLMs/AI tools should NOT be used for the planning or writing of articles in any way whatsoever. Using those tools for outlining purposes, as Benj says he has done, is not prohibited by the letter of your policy as shown in this retraction as far as I can see. The only exception I'd consider would be using an LLM as a glorified search engine to find links to actual sources. But the output of the LLM must not be used in any way aside from the links. And I think it's safer to just avoid them entirely.

Finally, we need a detailed post-mortem. How did this happen, and what changes are being made to prevent something similar from occurring again? While I understand that discussion of internal personnel matters is inappropriate, I do believe that this is serious enough to justify a change in staffing. Ars' credibility is on the line.

I'm posting my comment not having read to the end; I started my new job today (after more than a year of unemployment) so I'm arriving a tad late.

I think the original article and comment thread needs to be reinstated BUT the made-up quotes from AI need to be removed, and a note added indicating why there's a gap in the article. Additionally, the reposted article needs to have retraction explanation at the beginning of the article that explains what has happened and why the fake AI quotes are now gone. Then at least readers who never saw the original article can read what's left after the slop has been removed, and it won't continue to be slurped up by other AI entities, thus keeping the false quote floating around the interwebz.

 

[AI_Skeptic]

How would you want to be treated in a similar situation? Have you (and others) never made a professional mistake? And for those of you who have, and got the cold sweats when you realised, didn’t you swear to never let that happen again?

Journalism is so important, but nobody actually died from this mistake, and the scar tissue is invaluable.

Context matters. According to Benj Edwards website, he is the Senior AI reporter. What does this mean?

1. Out of all the journalists on Ars' staff, he is the expert on "AI". He should know that LLMs, like Claude and ChatGPT, are probabilistic word generators . In fact, according to his website, he's the one who coined the term "confabulation". Clearly, he used an LLM (which one? What's the tool name? We don't know, I can't find it) that falsely claimed to extract quotes from websites without questioning how such a tool could be possible.

2. He's a reporter (another word for Journalist). I do not know if Mr. Edwards has a degree in journalism, but his Ars Technica bio says he has over 20 years of journalism experience. He knows to always check quotes, because of how quickly people are willing to sue in the United States. That is something that is taught in the first day of a journalism class.

3. He has a duty, as a journalist, to check all quotes. Checking that quote requires a simple copy/paste to the website. He clearly did not do it, even though it could quickly be done.


This is not a "learning opportunity" where he could move forward on the AI beat. Either he does not understand how AI works, and therefore he should not be on the AI beat, or he does not do basic due diligence after 20 years of working as a journalist, and in that case he should not be a journalist, because he'll be too much of a liability for his employer.

I also question the "I have a fever and I tried using a new AI tool". When people have a fever, they revert back to the processes they are most familiar with, because their brains have less ability to learn new things. And even if a tool was used, it is quick to copy and paste the quote into the "find" bar in a web browser to confirm the quote is real.

 

[JMTronicHobbyist]

Here's my suggestion: Don't use AI tools to write, don't use them to "assist", don't even use them to summarize. A complete moratorium on AI writing or inquiries. Yes, of course I'd say this... but that this happened using AI was, frankly, inevitable. It's the nature of the tool, and it WILL happen again, even if it's writing is "proofread". The work it takes to verity each claim AI makes is better spent just doing that initial research and writing it with a human... by a human. You can if you wish grasp a whole other person I suppose, HR department may object.

No, no, no, the problem with all these AI debacles is that the people involved are not as Very Smart™ as I am. When I use AI I herd the cats with my own very special skills that no one else has.

 

[bodines1]

Benj being the person caught doing this is absolutely unsurprising. He has been writing mostly uncritical AI puff pieces for what feels like forever. Surely I'm not the only one that has been frustrated for a really long time at the quality of his work and the bafflingly uncritical nature of it?

 

[Ahabba]

I applaud ars for its stance on AI-generated material, and I'm sure this episode will be a valuable learning experience.

Meanwhile, I've come across the following op-ed in the Cleveland Plain Dealer, where an applicant withdrew an application for a reporting role after learning how AI is used extensively in this newsroom. And now the editor is chastising her and journalism professors for not being “prepared for the workforce.”

https://www.cleveland.com/news/2026...ear-of-the-future-letter-from-the-editor.html

Basically the guys at the plain dealer record their interviews and then have AI write the story, which they supposedly verify and double-check. He says: "By removing writing from reporters’ workloads, we’ve effectively freed up an extra workday for them each week."

I guess the stakes are lower in local journalism, but if this is the modern pipeline that young journalists are supposed to go through in order to eventually land better jobs, the future is looking bleak.

 

[AI_Skeptic]

Maybe I’m more lenient than most but I’m not mad at all about this. Errors in judgement happen all the time. I guarantee everyone commenting has had a serious mistake or two in their lives.

Errors in judgement should be forgiven. Errors in morals should not be forgiven. I do not believe that Benj made an error in judgement, but instead he made an error in morals.

1. He (SHOULD!) know, as an AI expert, that AI systems are probability generators, and can not think or make decisions. Because it's a probability generator, by definition, it can not extract quotes from websites. If there is such a tool created, that's worthy of an article itself.

2. He is a journalist of over 20 years. The first thing journalists are taught is to always verify the quotes. The quotes, in this case, come from a website, which could easily be verified using the "Find" command on any web browser. He failed to engage in that very simple task.

3. Because he used AI generated quotes, he put his employer at extreme legal risk. Thankfully the quote generated by AI wasn't harmful, and the person wronged took it in stride. However, there is no reason at all why he didn't verify the quote. But could it happen again? Would an employer want to take that risk?

Clearly, he is not a new journalist, and he is not new to AI. Either he's incompetent as an AI expert, and if so, dismissed from the AI beat - or incompetent as a journalist, and in that case he should be dismissed from working for Ars.

Why do you believe this is an error in judgement?

 

[AI_Skeptic]

Basically the guys at the plain dealer record their interviews and then have AI write the story, which they supposedly verify and double-check. He says: "By removing writing from reporters’ workloads, we’ve effectively freed up an extra workday for them each week."

I guess the stakes are lower in local journalism, but if this is the modern pipeline that young journalists are supposed to go through in order to eventually land better jobs, the future is looking bleak.

Wow! Talk about misunderstanding the purpose of a reporter! A reporter should select his words extremely carefully, to understand the impact of those words, and to trigger some type of emotional response in a reader.

I doubt that newspaper will last.

 

[TylerH]

"Everything in the article except the quotes" is a huge problem. It's even a huger problem that the quotes were from a website that could easily been verified using a copy/find/paste. It seems like the person who was incorrectly being quoted took it in stride, but what if he didn't? Condé Nast is a huge company, and if the person didn't take the quote in stride, they could have sued. It shows negligence on Benj's part not to do a quick verification that would have taken just a minute.

Like I said, at no point did I say there was not a problem. I wholeheartedly agree with folks that this is a major problem and one that should be taken seriously by Ars. I don't think the author should have used an AI tool at all, even if he were not dealing with brain fog or sickness and were otherwise able to have caught this mistake. I agree that it is a perfectly reasonable expectation for journalists to just read blog posts themselves and copy any quotes they want to use manually, the old-fashioned way.

One could, for example, look at my very first comment in the thread, from yesterday:

The only way to avoid this is to just not use AI, which is pretty easy; we all managed to do it before a couple of years ago. Many of us still manage to do it today.

My issue regarding this particular facet of the situation is and always has been what, from my perspective, appeared to be someone dismissing the entire retracted article as "AI[-generated] slop", when it was only some of the quotes that were AI-generated, not the entire article. Some people in this thread seemed to think that an insistence on getting the details correct meant siding wholly with the notion that AI-generated content being totally fine or that Benj did nothing wrong, which is silly and a very immature outlook.

As for lawsuits, I don't really think that is an issue; critically, one of the main requirements for defamation is reputational harm, but IIRC the hallucinated quotes did not harm the subject's reputation (they basically rephrased the spirit of the words the subject actually wrote on their blog, from what I can recall, and were about the dangers of AI, not statements of fact or the article subject's character or actions... recalling from memory here).

This also damages the likelihood of people wanting to be interviewed by Benj. For example, I wrote a book about AI. If Benj wants to interview me, would I let him? No. I do not trust that he's able to quote me without making mistakes.

Yep, I totally agree with this. Benj made that mistake, and owning up to it or not, he has to live with it. Whether Ars wants to keep him on given this likely diminished ability to access sources remains to be seen.

 

[r0twhylr]

That may have been a result of some closer over site of her work. Do you not remember the two articles she wrote early on (regarding NFT's if I remember correctly) that got torn to shreds in the comments similar to this one?

I don't, actually. Do you have a link for that?

 

[Marlor_AU]

1) Why didnt the review process catch the quotations that didnt actually exist? The whole point of a review process is to verify this exact sort of thing. There's defintely some major issues with the review process that did not catch this before publication, especially knowing that the author is someone who uses AI tools in their workflow for purposes of learning.

Once upon a time, this was the role of "sub-editors" or "copy editors". They would check the article for factual accuracy, readability and conformance to house style, and the article wouldn't go out the door until they were done.

However, in the rush to publish more articles at a higher cadence, the art of copy editing has pretty much disappeared. Even reputable newspapers often outsource it now to third-parties (for example, one company dominates the space here in Australia, with in-house subs being a rarity). There's a general consensus that these outsourced copy editors don't have the care, diligence or understanding of genuine copy editors, and are focused more on style than accuracy.

Conventional wisdom would consider this kind of editorial role as a prime candidate to be outsourced to LLMs entirely. But it seems to me that with increasing adoption of fallible models, human editorial oversight should be more important than ever.

 

[JMTronicHobbyist]

Good luck with that.

Maybe The Register, if you're a gentleman of culture? Not more responsible but possibly similarly?

 

[runswithjedi]

Context matters. According to Benj Edwards website, he is the Senior AI reporter. What does this mean?

1. Out of all the journalists on Ars' staff, he is the expert on "AI". He should know that LLMs, like Claude and ChatGPT, are probabilistic word generators . In fact, according to his website, he's the one who coined the term "confabulation". Clearly, he used an LLM (which one? What's the tool name? We don't know, I can't find it) that falsely claimed to extract quotes from websites without questioning how such a tool could be possible.

2. He's a reporter (another word for Journalist). I do not know if Mr. Edwards has a degree in journalism, but his Ars Technica bio says he has over 20 years of journalism experience. He knows to always check quotes, because of how quickly people are willing to sue in the United States. That is something that is taught in the first day of a journalism class.

3. He has a duty, as a journalist, to check all quotes. Checking that quote requires a simple copy/paste to the website. He clearly did not do it, even though it could quickly be done.


This is not a "learning opportunity" where he could move forward on the AI beat. Either he does not understand how AI works, and therefore he should not be on the AI beat, or he does not do basic due diligence after 20 years of working as a journalist, and in that case he should not be a journalist, because he'll be too much of a liability for his employer.

I also question the "I have a fever and I tried using a new AI tool". When people have a fever, they revert back to the processes they are most familiar with, because their brains have less ability to learn new things. And even if a tool was used, it is quick to copy and paste the quote into the "find" bar in a web browser to confirm the quote is real.

Clearly, he used an LLM (which one? What's the tool name? We don't know, I can't find it) that falsely claimed to extract quotes from websites without questioning how such a tool could be possible.

I searched with kagi for "claude code pull quote extractor" and found a tool that seems to fit Benj's description: https://github.com/nixlim/academic-quote-extractor

And here's a Reddit post of a student who used it instead of reading the assigned book.
View: https://www.reddit.com/r/ClaudeAI/comments/1qqtmct/academic_quote_extractor_cli_tool_for_pulling/


Gobsmacking phrases:

My uni is otherwise inclined, so reading full text end to end is not happening. I don't have space for all the violins in my head.

The idea is to use agentic LLM to do the reading and report back, with zero hallucination guarantee, the quotes. To make sure I get that guarantee, I get the LLM to bring me the IDs and scores for quotes. Quote text itself is always pulled verbatim from the database, never generated.

 

[Ragashingo]

Nearly the entirety of this thread has been emotional. Anger at how Ars has handled the retraction. Expressions of feeling "betrayed" by being fed misinformation. And most disturbingly, in my opinion, calling for the summary dismissal of an employee without a complete and thorough investigation.

This thread has become a horde of villagers with pitchforks standing outside of Frankenstein's castle, with a small contingent of other villagers at the gate saying "well, let's just wait a moment and evaluate this."

I'm curious, what is left to investigate?

1. Did he misquote someone? Yes.
2. Did those misquotes come about through the use of AI tools? Yes.
3. What is Ars Technica's stance on using AI material in its articles? Ken Fisher, Editor in Chief, says: "Ars Technica does not permit the publication of AI-generated material unless it is clearly labeled and presented for demonstration purposes."

These three things are known, and I believe they are enough to make a decision on Benj's fate. The question becomes: What does "not permit" actually mean? I'd be happy with a promise not to use AI again and a breakdown of how things will change at Ars Technica as a result. I also think it would be fair for Ars to part ways with an employee who broke fundamentals of journalism in such a stupid manner. But what we need most is a clearly made decision. Right now, it feels like... maybe the case is already closed? Certainly nobody has said it is ongoing. There has been no actual changes expressed. No punishments detailed. Do we just have to wait a week or two to notice that the writer in question hasn't written an article since this incident?? I kinda feel like we're owed a bit more than that!

There's also 4. Has AI really, really never been used in any other articles? That one is much harder to pin down and doing so would take significant time and ultimately still leave some unknowns. Aside from verifiably incorrect quotes, AI output kinda looks like human output and there's no good way to be sure which is which. Which is why the rule to not use AI in articles exists in the first place! Once you get caught, all your other work is suspect.

 

[runswithjedi]

Once upon a time, this was the role of "sub-editors" or "copy editors". They would check the article for factual accuracy, readability and conformance to house style, and the article wouldn't go out the door until they were done.

However, in the rush to publish more articles at a higher cadence, the art of copy editing has pretty much disappeared. Even reputable newspapers often outsource it now to third-parties (for example, one company dominates the space here in Australia, with in-house subs being a rarity). There's a general consensus that these outsourced copy editors don't have the care, diligence or understanding of genuine copy editors, and are focused more on style than accuracy.

Conventional wisdom would consider this kind of editorial role as a prime candidate to be outsourced to LLMs entirely. But it seems to me that with increasing adoption of fallible models, human editorial oversight should be more important than ever.

One of my favorite things about Ars is that they're often days behind the media frenzy to get a story out. I like to think they use that time to let the story cool off and develop interesting insights. I seriously hope that Ars management is not pushing writers just to get stuff out the door. I see it as a badge of honor for Ars to take days to put out an in-depth take.

 

[Sarty]

These three things are known, and I believe they are enough to make a decision on Benj's fate. The question becomes: What does "not permit" actually mean? I'd be happy with a promise not to use AI again and a breakdown of how things will change at Ars Technica as a result.

A promise to not use AI in direct and specific violation of policy again, plus two bucks, will get me a nice coffee at Starbucks.

Maybe three dollars now, damn inflation.

 

[Tennyson]

Ah, yes, the copy editor. Haven't heard that phrase in years, perhaps decades. No- one's willing to pay for what is perceived as a QC role.

The publishing equivalent of having someone else check your code.

Which ironically is how this whole situation started in the first place.

 

[AI_Skeptic]

As for lawsuits, I don't really think that is an issue; critically, one of the main requirements for defamation is reputational harm, but IIRC the hallucinated quotes did not harm the subject's reputation .

You're absolutely right, it's not an issue. In the USA, he will have to show harm if defamation happens, which clearly didn't happen...

Now, I'll put on my Risk Management hat and say this

"...This time"

Everyone got lucky in this case, the mis-quote wasn't defamation, and the guy who had the misquote attributed to took it in stride. But how do we know it won't happen again? Can Benj be trusted not to use some novel LLM tool to extract quotes from websites? What's the risk if Benj does mis-quote someone because of an LLM tool? And why, after 20 years, isn't confirming quotes before publishing automatic for Benj using an established process?

All questions Ars needs to ask.

 

[AI_Skeptic]

Clearly, he used an LLM (which one? What's the tool name? We don't know, I can't find it) that falsely claimed to extract quotes from websites without questioning how such a tool could be possible.

I searched with kagi for "claude code pull quote extractor" and found a tool that seems to fit Benj's description: https://github.com/nixlim/academic-quote-extractor

And here's a Reddit post of a student who used it instead of reading the assigned book.
View: https://www.reddit.com/r/ClaudeAI/comments/1qqtmct/academic_quote_extractor_cli_tool_for_pulling/

I saw that, and I THOUGHT there's no way he would use that. It seems so inefficient compared to using Python with Regex, or just opening up a PDF reader and searching by keywords and skimming.

If that's the tool he's using, that means he would have to save the website as a PDF, and run the PDF version of the website through the tool, and pray for the best. But given how small the website was, why bother using the tool anyway? He should have a notepad (the old fashion type), take notes on the website, and write down any quotes or interesting facts. Or if he's more modern, use Scrivener to capture all the relevant information (Scrivener is a non-AI tool!)

But what makes me raise my eyebrows is this:

To make sure I get that guarantee, I get the LLM to bring me the IDs and scores for quotes

Scores? Why would a quote need to be "scored"? Scored against what, and what is "relevance" in this case?

 

[Tennyson]

Scores? Why would a quote need to be "scored"? Scored against what, and what is "relevance" in this case?

Scored for relevance to the prompt, maybe?

 

[Chastilian]

I struggle to understand all the people who keep arguing that keeping the article online could be a negative because it could cause other LLMs to have access to bad information. The internet is FULL of wrong information. No person should ever let their actions be dictated by how it might effect the giant moneysink bullshit generators that are current LLMs. I would go so far as to say it might even be NOBLE to poison LLMs. If Sam Altman and his ilk cared about ensuring accuracy, maybe they shouldn't train their products on a torrent (no pun intended) of unfiltered and unverified content scraped en masse.

Either way, whatever action is to be taken should be judged on the merits it has for HUMANS, not products. Does the article staying up with proper context as a monument to a mistake serve the readership and the website better than the article being deleted? Is the subject of the article materially harmed by keeping the article up, or do they also oppose deletion? (hint hint, Ars should be asking Scott Shambaugh their opinion on this). Those are the questions to be asked. The fact that so many people even let concern for LLM 'quality' come to mind as relevant is quite frankly terrifying.

I agree with this. I cannot understand why so many people are siding with the decision to remove the article based on "LLM accuracy." We are in this position precisely because LLMs are fundamentally inaccurate; they already hallucinate and fabricate by design. Leaving a retracted article online with the proper context won't make these models any less reliable than they already are, but it is a major win for transparency and letting humans make their own judgment calls.

Removing the article is not the answer. It doesn't demonstrate journalistic integrity. It disrespects the community.

That said, I have no plans to cancel my subscription over this and I genuinely appreciate the work that ArsTechnica does on a daily basis.

 

[AI_Skeptic]

I've enjoyed Kyle Orland's work over the years but I won't trust it going forward and honestly will be moving on from Ars as well. This is not the first issue with intergrity they've had and at some point as a reader you have to admit there is something broken with the culture regardless of how much you enjoy the content.

Besides having his name attached (and from the sound of the article, wrote the first part), what's his role in this? Benj admitted the mistake, and that Kyle had no involvement. Why punishment Kyle?

 

[Resistance]

I'm curious, what is left to investigate?

1. Did he misquote someone? Yes.
2. Did those misquotes come about through the use of AI tools? Yes.
3. What is Ars Technica's stance on using AI material in its articles? Ken Fisher, Editor in Chief, says: "Ars Technica does not permit the publication of AI-generated material unless it is clearly labeled and presented for demonstration purposes."

These three things are known, and I believe they are enough to make a decision on Benj's fate. The question becomes: What does "not permit" actually mean? I'd be happy with a promise not to use AI again and a breakdown of how things will change at Ars Technica as a result. I also think it would be fair for Ars to part ways with an employee who broke fundamentals of journalism in such a stupid manner. But what we need most is a clearly made decision. Right now, it feels like... maybe the case is already closed? Certainly nobody has said it is ongoing. There has been no actual changes expressed. No punishments detailed. Do we just have to wait a week or two to notice that the writer in question hasn't written an article since this incident?? I kinda feel like we're owed a bit more than that!

There's also 4. Has AI really, really never been used in any other articles? That one is much harder to pin down and doing so would take significant time and ultimately still leave some unknowns. Aside from verifiably incorrect quotes, AI output kinda looks like human output and there's no good way to be sure which is which. Which is why the rule to not use AI in articles exists in the first place! Once you get caught, all your other work is suspect.

What good is a promise not to do something when that promise has already been made and broken once before?

 

[Sarty]

A red card does not mean booting the player from the team permanently - unless it was the result of deliberate malice.

No team - and you damn well know that - would boot a player permanently for getting a red card in a match.

If you incorrectly think that I am the one who introduced the soccer yellow/red card metaphor to describe Mr. Edwards' apparent misconduct and how Ars Technica might seek to discipline him, perhaps you unwisely used an AI tool to summarize the comment section up to that point.

 

[acefsw]

Even bolding key points of quotes in the forums gets a textual slap from Ars with threats of bans for “manipulating” quotes. We should keep that in mind when we observe the response to this

Touché.

 

[dwl-sdca]

If a car mechanic took a hammer to your engine block and then said "I was sick" when confronted with what he did, would you ever take your car back to that mechanic?

I’m replying with 4 more pages of comments left for me to read but I can’t help it. At the time of this reply there are 21 downvotes to this comment. I can’t understand it. I think that the image of a crazed mechanic is less a professional betrayal than a journalist falsifying a quote. It is a fundamental part of a journalist’s responsibility to verify their quotations. I believe that the journalist should often go further and ask if the quoted person has a further comment.

Please, someone here explain how at least 21 Ars readers can think it is okay for a journalist to misrepresent a key quotation. To my thinking, once is enough for a true journalist to post that kind of lie. That the quote was more benign than, some vulgarity doesn’t matter. If a journalist wants to make something up they should change careers and write fiction. A journalist’s only job is to find useful information and to report the facts. Sometimes it is proper to clearly offer an opinion about the implications of those facts. Distortion of facts is simply unacceptable.

Edit: I’m trying to not jump upon a table, wave my arms and shout. The problem is the unverified quote. That Edward’s may have violated an Ars policy concerning AI is a distraction.

 

[modelbrains]

What good is a promise not to do something when that promise has already been made and broken once before?

well now that they got caught they'll make extra-certain they don't get caught again

 

[Woolfe]

I haven't read all 28 pages of controversy.

I did read the Screenshots of Benji's comments, and detail on the incorrect quotes.

Ars did the right thing by Retracting the article. BUT they should be working on a way to bring the article back with details around how it was wrong and how they will address the problem.

This appears to be an administrative failure brought on by an individual who shouldn't have been working if they were sick.

This will tarnish both Ars and Benji's reputation. Which is a shame, because they do have controls in place but they have failed, which is immeasurably better than many other entities who just publish the wrong information and don't care.

I'd rather stick with Ars who recognised they made a failure, took action to fix it, and let everyone know they made the mistake.

 

[Dicere1]

Ok. Go ahead, what's your name and address? Be the change you want in the world.

Your error is in thinking it will be a matter of personal choice. Stylometry and metadata triangulation combined with AI being cheap for the consumer and the recursive nature of AI training mean that the internet will not be anonymous for long at all. But the repercussions of clawbots mean that eventually someone will have to take legal responsibility for their actions.