Editor’s Note: Retraction of article containing fabricated quotations

We are reinforcing our editorial standards following this incident.

See full article...

 

[adrianovaroli]

Five stages of grief but now it's about quoting comments.

As for the "APA says X", while that may be true... These are comments sections.

 

[AdrianS]

I'd say let sleeping dogs lie.

But this is the internet!

 

[Wheels Of Confusion]

He also links to the post-incidence blog post from (supposedly) the real human owner of the agent.

Bunch of admissions in the form of questions there.
It boils down to "There may be consequences, but none of them are for me so does any of this even matter? (Other than that I want to do it, of course.)"

As for the "APA says X", while that may be true... These are comments sections.

There are times when Ars has imposed rules on commenters that are more strict than the standards they hold their own authors to, and that should be an indication that somewhere the plot has been lost.

 

[Thakk]

That said, curiously enough, I actually have The Man From Earth (and its sequel, though I've yet to watch either: my backlog is absurd) on Blu-ray rather than DVD. it's definitely out there. (I also still have the DVD I never watched.

The sequel is awful. There should have been only one.

 

[BernieW]

If I use a tool to generate quotes for an article, then manually verify each quote (presumably by copying it and searching for it in the source), how is the tool valuable? It didn't save any time, and it introduced the risk of fabricated quotes, a risk that wouldn't have been present if I didn't use the tool.

For searching a short blog like this, there's probably no time savings to be had. For a many pages of text, it could save a lot of time. For example, once I was searching for something I half-remembered in the Columbia Accident Investigation Report. A conventional search turned up nothing since I could only remember the gist of it. I ended up scanning through the whole report to find what I was looking for. In that case, an AI tool might have saved me a lot of time if it could have found something based on my fuzzy description. I would still have to use a conventional search on the exact text the tool gave me to verify that it wasn't an AI hallucination.

 

[Wheels Of Confusion]

We'll have to disagree about that one. And your comments indicate you should not be given the benefit of the doubt either, which was my entire point. Some things, the person doing them must stay above reproach.

Please spare us all this hypocritical prudishness. Must all school crossing guards also strictly use sex for procreation, and only if adhering to the missionary position?

The fact that your original post is unanimously downvoted two and a half dozen times as of this writing is a suggestive indicator that your knee-jerk reaction is way out of norm with most of your peers. That you can only draw the conclusion that nobody else here is actually one of your peers but must somehow be deficient in life experience and perspective reinforces the impression that you're unreasonably judgmental.

 

[HoorayForEverything]

Not to jump on this post too much, but the AI ethics discussion ended in 2020 when Google fired their AI ethics department for doing ethics. There’s nothing to write about.

That ends the discussion for Google. What about the rest of the industry? Part of this week's buzz has been around an ethics piece from an ethics member of staff at Anthropic, who left OpenAI over ethics. Quite a lot of the buzz has been discussion of a shitty headline from the New Yorker above the piece, from people who didn't read the piece.

Historically I've come here to avoid that sort of nonsense and this is perhaps an opportunity for Ars to get back to providing a better view of this sort of thing.

(I really wish I didn't have to indulge "this week's buzz" but it's likely to come up in the day job at the mo. As soon as I have completed my cunning plan to find a part of the industry where I can just not engage with the daily rumour-mill, I'll revert to not giving a shit.)

I think the self hosting article would be interesting; my experience has been that it’s mostly a waste of time, money, and intellectual capacity and I’d like to see something on the topic that isn’t just a bunch of people posting their very expensive collections of GPUs on Reddit for fake internet points.

I think this hobbyist version is a bit silly, agreed, but there are multiple studies which show there's a decent tipping point now which means you can train it on sensitive data and avoid a whole bunch of governance and grind because this way the entire enterprise remains on-prem. It's become clear that for cases where you want to feed an LLM your entire enterprise and ask it daft questions, the last 5% of utility and accuracy from hyperscale provisioning isn't relevant, so there's a very strong 95/5 case, not even an 80/20 case, for doing this.

Corporates have spent 50 years trying to get "knowledge management" right and still haven't, and they probably never will because humans are just shit at classifying and summarising things, with a big force multiplier on not caring, because most of the really valuable tradecraft is from your most demotivated and underpaid field staff.

So a magic free-form tool that works 95% of the time absolutely is a fucking miracle for some use cases.

And I agree nobody is writing about that. TL;DR: I want to slap everyone at TLDR with a wet fish.

PS it is becoming clear I have not, in fact, left, but I seem to be time-locked on changing my username.

[Edited for multiple missing keystrokes]

 

[adrianovaroli]

Sorry, everyone. I'm a little behind on the comments here. There are 39 pages!

Anyway, what you're saying here is that LLM's don't really think, they don't really 'reason', they just come up with a text response based on the human interactions/writing it was trained on.

What it came up with was what could only be taken as threats, blackmail, and/or extortion.

What does that say about humanity?

That we are a species prone to exaggeration. Look, yeah, it's not good, but the vast corpus of human art does include writing, and filming, and singing, about threats, blackmail and/or extortion. The "murder ballad" is a whole genre. So when you feed an LLM as much as you can it will also incorporate that. And when you prompt an LLM with "do this and then blog aggressively", or worse (not considering the fact that it might have even been some actual jackass doing it and pretending to be an LLM, which is also being discussed), you do get this kind of output.

 

[Wheels Of Confusion]

Jeff Geerling posted a short video about the entire AI situation sparked off by Shambaugh's situation. But it leads off with the bit about Ars retracting the article due to AI hallucinations.

View: https://www.youtube.com/watch?v=bZJ7A1QoUEI

 

[adrianovaroli]

nvm, fucked up an edit

 

[ERIFNOMI]

"Crossing guards can't have casual sex, they must be held to a higher standard" is not a discussion I expected to see here.

Where do you people come from?

 

[carldjennings]

The sci-fi of my youth got it wrong. We were meant to have "AI" that had infallible, always-accurate knowledge, but struggled to articulate it in anything but a robotic monotone. Instead, we get "AI" with lossy and inaccurate knowledge, but which expresses it with such articulation that we trust it anyway.

Man, we did not watch the same sci-fi movies at all. I remember HAL, WOPR, and SKYNET. I don't know what you watched.

 

[SomewhereAroundBarstow]

When I hear the phrase "sex tourism" used, it's more frequently applied to Thailand, the Philippines, and Russia.

It's a ridiculously loaded thing to talk about on the internet, as the reply from the poster who first brought it up and is now insinuating that my objection to "if A, then B" assumptions about people suggests I have something to hide shows. But that said the first place that pops into my mind when I hear the term is Amsterdam. And just to reiterate what I said before about myself no, I've never been to Amsterdam. I lead a pretty dull life and that's okay with me.

 

[Readercathead]

I’m still checking every day for that update we were promised.

 

[macosandlinux]

No, you didn't. The rule is no modification of the quote you select, not that you must quote the entirety of the post.

That rule IS pretty draconian--it's been established that so much as bolding a word in the quote and adding (emphasis mine) just beneath the quote is a rule violation, as is [paraphrasing in square brackets]. But it does not prevent you from only quoting the bit you mean to reply to.

Good to know, I did bold sometimes key words that I wanted to highlight. Won't do it again

 

[Jim Salter]

And just to reiterate what I said before about myself no, I've never been to Amsterdam.

I have. No hanky panky, though, just delivering a talk, visiting a tea shop and a bunch of museums. The Netherlands is pretty lovely!

 

[Dark Jaguar]

"Crossing guards can't have casual sex, they must be held to a higher standard" is not a discussion I expected to see here.

Where do you people come from?

They are anointed by the High Reactionary, who doth sayeth that all shall be defined by their job, and their job shall define them, such that none may cross the road to the path of darkness, lest that taint be returned to the children and their children's children, for three generations.

 

[macosandlinux]

It sounds like Benj chose to work from home rather than just taking a sick day. Maybe he didn't feel that sick, but wanted to avoid exposing anyone else.

Ars writers are always remote.

 

[Readercathead]

I love Ed, but I'm not sure Conde Nast would be happy with 10,000 word pieces filled with delightful swearing

It works for the Verge! They’ve built a very loyal readership and membership base that way. No holds barred criticism of anyone and everyone plus lawyerly perspectives on the intersection of tech and policy. They have a very clear voice and obviously do quite a lot of work editing and supporting the writers, I’m not sure Condé Nast is interested in making that kind of investment in journalism. A dozen lone bloggers left to twist in the wind on their own seems to be what we have here and it’s fraught with its own kind of peril.

 

[HL7]

For searching a short blog like this, there's probably no time savings to be had. For a many pages of text, it could save a lot of time. For example, once I was searching for something I half-remembered in the Columbia Accident Investigation Report. A conventional search turned up nothing since I could only remember the gist of it. I ended up scanning through the whole report to find what I was looking for. In that case, an AI tool might have saved me a lot of time if it could have found something based on my fuzzy description. I would still have to use a conventional search on the exact text the tool gave me to verify that it wasn't an AI hallucination.

I was referring to generating quotes from known source material, not searching for unknown material. Giving it known material, having it generate quotes, and then manually checking each quote takes as much time as extracting the quotes yourself.

The safe and actually time-saving version of this -- if you had a really long text to find quotes in -- would be to ask the AI to find what it thinks are the best quotes and give you the first few words so that you can go search and extract them. You're outsourcing the decision about which quotes are relevant to AI, and it might pick bad quotes, but you'll at least have real ones, and it's probably faster than hunting through the text yourself. Whether that kind of tradeoff is worth it is up to the individual author.

I have used AI for fuzzy search and it certainly can be a time saver ... the only thing my wife could remember about a recipe was that it called for exactly 3 red peppers, and AI found it in the first hit. But there are no big drawbacks there -- if the AI hallucinates, I just didn't find what I was looking for, which is what was happening anyway.

They call it generative AI, but actually generating things is often the worst way to use it.

 

[SomewhereAroundBarstow]

I have. No hanky panky, though, just delivering a talk, visiting a tea shop and a bunch of museums. The Netherlands is pretty lovely!

I'd love to go someday.

The closest I've ever come to the topic at hand was when I was killing a couple hours wandering in Helsinki before going to the airport. I came across a storefront with curtained windows and the universal sign for "cup of coffee" in one of them. The Finnish sign might as well be Greek to me so I opened the door expecting to find a coffee shop. Instead I found racks of pornographic magazines and four or five men standing around, each with one open in their hands. Their heads turned as one to look at me, as did the man at the counter. I backed out and decided it was time to head back to the hotel.

 

[Thad Boyd]

Then no, I probably didn't. You gonna tell, or?

It breaks the exact same rule you did.

Hey, the rules may be absurd, but at least they're inconsistently enforced.

 

[Dark Jaguar]

Ars writers are always remote.

I feel like that's the ideal environment for writing as it is, though less ideal if one needs to do some original research and might need to interview someone or check out something from the library or a records office. I don't expect there's a lot of Ars topics requiring clandestine meetups with shadowy figures in a parking garage in D.C. though.

 

[ArsLongaVitaBrevis_4321]

Hmm, let me check my notes . . . We’re now one and a half days past a holiday; and two and a half days past the weekend. Will we ever see an update on this incident??? And by update, I mean a proper and complete explanation! (Yes, I know that this can’t happen instantly, but it should be Ars’s top priority, IMO.)

 

[FangsFirst]

The sequel is awful. There should have been only one.

I feel like I have indeed heard this—some year I'll find out if I actually agree!

 

[macosandlinux]

I feel like that's the ideal environment for writing as it is, though less ideal if one needs to do some original research and might need to interview someone or check out something from the library or a records office. I don't expect there's a lot of Ars topics requiring clandestine meetups with shadowy figures in a parking garage in D.C. though.

Well, there's the policy, security and health sections that might require discretion with sources, for example.

 

[Dark Jaguar]

For searching a short blog like this, there's probably no time savings to be had. For a many pages of text, it could save a lot of time. For example, once I was searching for something I half-remembered in the Columbia Accident Investigation Report. A conventional search turned up nothing since I could only remember the gist of it. I ended up scanning through the whole report to find what I was looking for. In that case, an AI tool might have saved me a lot of time if it could have found something based on my fuzzy description. I would still have to use a conventional search on the exact text the tool gave me to verify that it wasn't an AI hallucination.

I think something of better use would be for authors to provide an index or appendix of longer works, as is common in books at least. Some helpfully provide a list of all their quotes at the end of their articles, or an index including links to sources. I know wikipedia isn't a primary source, and so do they, which is why that's standard practice in their own articles.

 

[Dark Jaguar]

We'll have to disagree about that one. And your comments indicate you should not be given the benefit of the doubt either, which was my entire point. Some things, the person doing them must stay above reproach.

I deliberately chose a slightly-off-center example to demonstrate what "above reproach" means. When you do something questionable in your personal life, like having an OnlyFans, it can lead to losing your job. That's just life.

Now consider a dead-center ethics violation like what Benj did in the article. It's 100x worse than my deliberately-off-center examples. So, it's pretty much unforgivable out of the gate.

What's "questionable" or "reproachable" about that? It's not a matter that impinges on anyone's rights or is harming anyone in any way. It's not my business if a teacher has some sort of side business of that nature, not at all. Don't bring it to work, there's no problem. You've taken "above reproach" to mean something beyond the original scope. You definitely want someone to not do anything that even has the appearance of impropriety, with STUDENTS, but outside of that environment, with other consenting adults? No one's business, not even suspicious.

 

[TylerH]

They call it generative AI, but actually generating things is often the worst way to use it.

They call it generative AI because that's the only form of output it's capable of. I don't use any genAI tools, but from what I know they all only generate text or images in response to prompts. They don't take you to the page of a document you're searching and highlight matches like a Ctrl+F search function does, do they?

 

[timby]

Why are you so hungry for blood? What have you ACTUALLY suffered that justifies destroying someone’s livelihood over a policy infraction? The comparisons to vehicular manslaughter while drunk are completely asinine and, frankly, make me think that a large number of the commentators need to disconnect from the internet for a while and quietly reflect. You choose to get drunk. You don’t choose COVID induced brain fog. I think there’s a fair amount of implicit ablism here that needs to be interrogated.

Here's the thing, and I say this as a recovering journalist: We, as journalists, are held to a standard, because we have a duty. That duty is to the truth, because it is our job to report the truth, accurately, because we are the record.

I know that's hopelessly naive in the era of "fake news" and whatnot, but it's what I was taught in J-school, and that's how I worked as a writer and editor.

When generative AI gets brought into the workflow and is contributing materially to that record, the truth is in jeopardy, and the record itself is in jeopardy.

I don't know Benj at all, and to my knowledge, I've never interacted with him. But he knowingly filed a story with content that had been processed and output from an LLM. That represents a significant dereliction of duty, and it does not meet the standard of journalism in any way.

It's not a matter of being hungry for blood, at least for me. It's that Benj poisoned his own credibility as a journalist, and Ars' credibility as an outlet of the truth is poisoned by association with him. That is the reason some, or many, people believe termination is the only viable way forward, because Ars is at a very real inflection point here regarding its future as a respected source of the truth.