Editor’s Note: Retraction of article containing fabricated quotations
- Thread starter JournalBot
- Start date
It was caught because the author of the blog post, Scott Shambaugh, posted in the comments that the 2nd half of the article was made up, that the quotes attributed to him were fabricated. Other posters who had read his blog post, affirmed his statement was correct.
I'm glad ars acknowledges the error, but removing the article and shutting down all commentary strikes me as counter productive, as far as transparency goes. It's not like there haven't been other controversies before.
Upvote
320
(323
/
-3)
I just want to say on a personal note that I really appreciate our readers. I try and keep up with our comments, but I can't read them all, especially in real time. I do rely on people reporting issues to make sure they don't slip through the cracks.
I got reports about Scott's post and because of that I was able to immediately bring the matter to people's attention internally. I probably would have seen it eventually, but by when?
It was a Friday afternoon, it's a holiday on Monday in the US, and in some places kids have the entire week off from school and there are road trips etc. That is ... not great timing.
So yeah. Thanks for the team effort, this community had our back.
I got reports about Scott's post and because of that I was able to immediately bring the matter to people's attention internally. I probably would have seen it eventually, but by when?
It was a Friday afternoon, it's a holiday on Monday in the US, and in some places kids have the entire week off from school and there are road trips etc. That is ... not great timing.
So yeah. Thanks for the team effort, this community had our back.
Upvote
550
(571
/
-21)
- Scott Shambaugh's actual own words - https://theshamblog.com/an-ai-agent-published-a-hit-piece-on-me-part-2/
The misquote makes it even more ironic. I cancelled my sub yesterday after I learned of this, and it will take time to gain back my trust.
Upvote
65
(128
/
-63)
The whole AI situation is so strange. I have started using Gemini quite a bit in the last 3-4 weeks, and it is shocking how good it seems to be at first, and how much detailed information it can give me about obscure topics like particular revisions of automotive parts or configuration settings.
What's also eye-opening is just how often it's completely wrong when you go to verify some of that information.
...and when it's incorrect, it's confidently incorrect.
It has suggested parts that fulfill my requirements that simply don't exist, and it has explicitly told me which programming values to change to update the 12V battery configuration in my Mach-e, and when verifying, those values are in the wrong location.
It even explains why those are the correct parts or correct values, and tries to convince you that it's correct.
When I correct it, it says, "well spotted! Those are the correct values for the F-150 and x other vehicle. The correct value for your vehicle is 'y'".
It's often still incorrect.
It's so important to verify these things. I would say it's never appropriate to use this to pull factual information for a journalism piece, unless it's just for a search, and then you go to the primary source.
EDIT: Changed my wording to shocking "how good it seems to be at first". As others pointed out, it can't be incorrect and still be shockingly good. it's just uncanny how much information it can gather from so many sources at once. That makes it a pretty good search engine, but when it tries to draw conclusions and give you a definitive answer, it often falls flat. Also clarified that this is not appropriate for journalism.
What's also eye-opening is just how often it's completely wrong when you go to verify some of that information.
...and when it's incorrect, it's confidently incorrect.
It has suggested parts that fulfill my requirements that simply don't exist, and it has explicitly told me which programming values to change to update the 12V battery configuration in my Mach-e, and when verifying, those values are in the wrong location.
It even explains why those are the correct parts or correct values, and tries to convince you that it's correct.
When I correct it, it says, "well spotted! Those are the correct values for the F-150 and x other vehicle. The correct value for your vehicle is 'y'".
It's often still incorrect.
It's so important to verify these things. I would say it's never appropriate to use this to pull factual information for a journalism piece, unless it's just for a search, and then you go to the primary source.
EDIT: Changed my wording to shocking "how good it seems to be at first". As others pointed out, it can't be incorrect and still be shockingly good. it's just uncanny how much information it can gather from so many sources at once. That makes it a pretty good search engine, but when it tries to draw conclusions and give you a definitive answer, it often falls flat. Also clarified that this is not appropriate for journalism.
Last edited:
Upvote
355
(366
/
-11)
I'd be interested in reading a follow up price from the original article author(s)... even if it's the last thing that they publish on Ars.
I guess we'll see what sticks once the tree finishes shaking.
I guess we'll see what sticks once the tree finishes shaking.
Upvote
168
(170
/
-2)
I sincerely appreciate that Ars pulled the article and issued a retraction @Aurich.
In the interest of full transparency and trust, though, I think it's important that the authors of the article, who appear to have used AI themselves to hallucinate quotes and attributed them to the subject of the article, openly discuss and acknowledge what they did. Unless Ars simply plans to fire them/not accept further contributions from them.
The original article is attributed to Benj Edwards and Kyle Orland. The fact that the article did not meet the journalistic standards of Ars and got retracted is important. But the stance of the authors is equally important because it is a reflection on their future contributions and whether we should trust them.
In the interest of full transparency and trust, though, I think it's important that the authors of the article, who appear to have used AI themselves to hallucinate quotes and attributed them to the subject of the article, openly discuss and acknowledge what they did. Unless Ars simply plans to fire them/not accept further contributions from them.
The original article is attributed to Benj Edwards and Kyle Orland. The fact that the article did not meet the journalistic standards of Ars and got retracted is important. But the stance of the authors is equally important because it is a reflection on their future contributions and whether we should trust them.
Upvote
556
(560
/
-4)
So the issue is really that Ars used AI hallucinated quotes but the whole AI agent attack and writing a defamatory article actually happened. At first when reading the retracted notice and replaced text in the original ars article, I understood it to mean that the WHOLE thing never happened.
Upvote
179
(179
/
0)
What's being done to/about the authors?
From what I can see, they're both still Ars affiliated:
https://meincmagazine.com/author/benjedwards/
https://meincmagazine.com/author/kyle-orland/
From what I can see, they're both still Ars affiliated:
https://meincmagazine.com/author/benjedwards/
https://meincmagazine.com/author/kyle-orland/
Upvote
130
(162
/
-32)
I feel like "uphold" is a pretty strong word for what this is. They violated them, and said "we violated them." I will grant that they reiterated their standards that were breached.
They just said "oops" and "we won't do it again."
I would say that this is not transparent. It is as opaque as it could possibly be. We already knew that the article had fabricated quotes, thanks to the subject of the article. We didn't know who did the fabricating, and what the consequences would be for them. The only people who didn't know were the ones that didn't see the article, since they took it down rather than leaving it up and putting a notice at the top saying it likely contained fabricated quotes and was under investigation.
I would hope to see actual transparency next week (and totally understand that humans need time off) before I consider this to be addressed properly.
It is pointless to have rules if there are no consequences for breaking them. We're seeing that play out on a daily basis both in the US and the world. If you want to keep that all an internal matter, that's your prerogative. But there will be consequences of that among your readership.
Last edited:
Upvote
292
(322
/
-30)
So at this point it seems a little bit like OpenClaw and the like are mainly written to be trolling tools. Like seriously, set AI loose on the internet with all the fine examples of behavior, what do you expect? Is there a little bit of glee that things go off the rails?
Upvote
72
(72
/
0)
ubercurmudgeon
Ars Tribunus Militum
That this has happened here shows how insidious the temptation to use AI as a shortcut is, like root beer, and the Federation.
Upvote
171
(175
/
-4)
What happens with the authors going forward will have a direct impact on if I resubscribe or not. If there are no consequences for this irresponsible and unprofessional behavior, I can't support that. I'm not too worried about that, though, Ars management seems to have a good head(s) on their shoulders. Please choose trust- readers' trust that is a truthful source- over loyalty to employees who are violating that trust
Upvote
160
(187
/
-27)
Thank you Ars. I was disappointed like everyone else and kind of horrified but this is a strange new world we live in and mistakes do happen sometimes and I am more than willing to give you guys the benefit of the doubt because I know you are taking this seriously. The article in question was very quickly retracted, investigated, and now apologized for.
And I can live with that as long as it’s an isolated incident! I’ll be keeping my subscription. Thank you for the quick response and action and I hope this whole episode ends up being a good thing that strengthens editorial standards.
And I can live with that as long as it’s an isolated incident! I’ll be keeping my subscription. Thank you for the quick response and action and I hope this whole episode ends up being a good thing that strengthens editorial standards.
Upvote
42
(67
/
-25)
This is a completely unprofessional retraction, unlike any I've ever seen, that doesn't even name the article being retracted.
It is a good step but it's not meeting journalistic standards either.
Upvote
511
(524
/
-13)
I hope to see a fuller response from Ars as to what happened and what the full outcome is. But PLEASE, as readers, let’s not speculate on specific individuals and what their role might have been. It’s not fair to anyone to prematurely impugn individual reputations and doesn’t advance the situation in any way. There were two authors and we currently have no way of knowing what their relative responsibilities were.
Upvote
67
(109
/
-42)
Two comments:
First, dropping your subscription because a single author violated Ars policy and wasn’t caught before publication seems excessive, particularly considering they acted aggressively over the weekend and even admitted precisely what was wrong.
Second, it gets tricky when possible employee discipline is involved. I think that has to be handled first, before additional public postmortem.
I don’t like it, but I’m also not aware of any publication acting more aggressively and publicly than Ars has (so far) in a similar case.
First, dropping your subscription because a single author violated Ars policy and wasn’t caught before publication seems excessive, particularly considering they acted aggressively over the weekend and even admitted precisely what was wrong.
Second, it gets tricky when possible employee discipline is involved. I think that has to be handled first, before additional public postmortem.
I don’t like it, but I’m also not aware of any publication acting more aggressively and publicly than Ars has (so far) in a similar case.
Upvote
317
(386
/
-69)
I'll expect nothing short of a full, deep-dive explanation from Ars in a few days. This is quite the event.
Upvote
240
(245
/
-5)
I'm not dropping my subscription, but I'll be keeping an eye on the situation and if there are no consequences for the authors over the next few months, I won't be resubscribing. I agree that cancelling at this point would be hasty, but if it's just business as usual after this and the authors continue to publish, I can't trust them or this site to be a truthful source. I'm paying because I believe Ars to be a truthful source worth supporting. If it's not, I'm out
Upvote
192
(206
/
-14)
The only thing I don't like about how Ars has handled it so far is that they didn't call attention to which article, and left it to the comments to find which one was retracted. If this were an article that I read and formed opinions on, I'd certainly like to know so that I can re-evaluate those opinions with new information.
Upvote
337
(338
/
-1)
As my father used to say "It takes 10 rights to right a wrong".
Once that trust is lost, it's nearly impossible to get back. Public trust is a finicky and finite thing.
Upvote
90
(100
/
-10)
I don't understand commenters commending Ars for the statement. It's the bare minimum and reads a little like a dodge. Maybe they need time to figure out how this happened, but until they explain the scope and nature of the issue, how do we trust anything we read here? The source blog takes 10 minutes to read. Why was an AI summary even necessary?
Ars' reporting on AI has felt like sane-washing. I understand everyone is excited about AI, but Ars doesn't approach AI with a healthy skepticism. That used to be what set it apart from the sea of tech booster infomercial websites. But recently, It's just, wow, Chrome AI agent went to the PS5 website and slowly sorta did a very simple task, 7/10. And now this.
Last edited:
Upvote
231
(266
/
-35)
When Stephen Glass was found to have fabricated a story for the New Republic, there was a long investigation of prior stories and quite a bit of public disclosure. Will that be happening here?
Edited to add: I don't think this situation is equivalent to Glass's conspiracy, and this was probably not the comparison to use. But Christ, what a pointless way to throw away one's credibility (and the credibility of one's employer.)
Edited to add: I don't think this situation is equivalent to Glass's conspiracy, and this was probably not the comparison to use. But Christ, what a pointless way to throw away one's credibility (and the credibility of one's employer.)
Last edited:
Upvote
201
(209
/
-8)
I hope we get a new article (100% written by humans) on the original subject. I still have doubts at just how much "autonomy" was used by the AI agent in the pull request and subsequent blog post.
As for using AI without disclosing it, the authors of the article need to step up and explain what happened if they are to continue to publish here at Ars. If they don't and continue to work here, I won't be reading their articles.
As for using AI without disclosing it, the authors of the article need to step up and explain what happened if they are to continue to publish here at Ars. If they don't and continue to work here, I won't be reading their articles.
Upvote
129
(130
/
-1)
Please elaborate - exactly how can fabricated quotations be published in a manner consistent with the policy?
Upvote
42
(92
/
-50)
This is why such notices need to disclose that a port-mortem WILL happen.
Ars does not have a great track record with post-mortems when breaches of journalistic integrity have happened. And this is far from the first happened-just-before-the-weekend emergency fire over breaches of integrity in their coverage.
This is why we need more transparency.
I'm willing to wait a few more days because this might involve personnel adjustments and that will take time and work-days to happen. But this retraction notice cannot be the end of it. That's just not enough.
Upvote
279
(283
/
-4)
I appreciate Ars editorial acting quickly and publishing the apology on the Sunday of a holiday weekend.
There still is the question of how the original article came to be written and published with false content. It is not just that some of the content was AI generated. It was wrong, and not fact checked.
Scott Shambaugh's blog posts are excellent. He raised a vital issue that deserves better analysis and discussion.
There still is the question of how the original article came to be written and published with false content. It is not just that some of the content was AI generated. It was wrong, and not fact checked.
Scott Shambaugh's blog posts are excellent. He raised a vital issue that deserves better analysis and discussion.
Upvote
149
(151
/
-2)
This is where I'm at as well. I'll be keeping an eye on my RSS feed for an update on this situation and how Ars intends to move forward before making any decisions.
Is Ars written policy on AI use in articles available for the public? I tried to search for their policies and was unable to find them. Some other outlets that I support have their editorial standards publicly available, many of which explicitly prohibit the use of AI in reporting without rigorous review and disclosures.
Upvote
112
(112
/
0)
I mean, if they were doing an article about how AI can fabricate quotations, I guess it could be consistent with their policy to then use AI to fabricate quotations. Obviously that's a contrived example though...
Upvote
52
(52
/
0)
I simply do not find this statement to be edifying or satisfactory enough over what transpired on Friday. Like others here, I'll give the Editors a couple more days to decide if they want to elaborate further before I make further decisions on how I engage with Ars Technica.
Upvote
114
(123
/
-9)
I'm pleased by the retraction and notice, but displeased by the approach. I'll echo others in saying I expect the retraction itself to include the title of the original story being retracted (and ideally, a link as well); and I think it reasonable to also expect that the original story will not be removed, but rather altered with a prominent notice of the retraction at the top of the story, ideally with a link back to the retraction notice.
Anything less leaves one with the lingering suspicion, however undeserved, that the editorial staff is more interested in "burying" the mistake than correcting it.
Anything less leaves one with the lingering suspicion, however undeserved, that the editorial staff is more interested in "burying" the mistake than correcting it.
Upvote
209
(212
/
-3)
Which is why I am giving them the benefit of the doubt. This was a really bad event for them. I can’t imagine anyone there being happy with this. I’d imagine they are going to do some serious crackdown on making sure this doesn’t happen again because it’s just about the most damaging thing they could do. I know they’ve lost subscribers over this and that many people will not give them another chance.
I’ve been there. I loved Washington Post until they shit the bed and I had to cancel. I don’t see that happening here, at least for me. I can’t imagine them doing anything other than taking this very seriously. They know this could be an extinction level event for a small publication focused on technology.
But should I be wrong and it happens again I’ll be lining up to cancel too. I just think they’ve earned a little bit of understanding and patience, more than any other media outlet I’ve ever interacted with at least in my view.
Upvote
170
(173
/
-3)
Here's my suggestion: Don't use AI tools to write, don't use them to "assist", don't even use them to summarize. A complete moratorium on AI writing or inquiries. Yes, of course I'd say this... but that this happened using AI was, frankly, inevitable. It's the nature of the tool, and it WILL happen again, even if it's writing is "proofread". The work it takes to verity each claim AI makes is better spent just doing that initial research and writing it with a human... by a human. You can if you wish grasp a whole other person I suppose, HR department may object.
Last edited:
Upvote
243
(247
/
-4)
Resistance
Wise, Aged Ars Veteran
Agreed, I am absolutely willing to give ars some time to produce a quality response, especially considering the weekend timing of the incident. The response so far is sufficient as a temporary measure.
Upvote
141
(146
/
-5)
- Status