Steven Rosenbaum explains how inaccurate quotes got into his book <em>The Future of Truth</em>.
See full article...
See full article...
AI put “synthetic quotes” in his book. But this author wants to keep using it.
- Thread starter JournalBot
- Start date
Sure, the author maintains the final responsibility for whatever's in the book. He's the one that requested the AI help that led to the "synthetic quotes" being published, and didn't catch the error in what he got from the AI.
All that said "His use of AI research tools led to him missing "synthetic quotes" that got inserted into his book" is a little long for a headline...
Upvote
11
(11
/
0)
I’m reading this thinking “this man’s brain is cooked.” The tool is making stuff up, and making him look silly and stupid for putting them in a book, and he doesn’t want to go back to the way things were, when he was less likely to have made-up quotations in his work?
Upvote
25
(25
/
0)
Lying liar gets caught lying, makes terrible analogies, concludes his non-mea culpa by promising he's learned nothing at all.
He is an idiot, delusional, a liar, and just based on how he comported himself here, a bad writer besides.
He is an idiot, delusional, a liar, and just based on how he comported himself here, a bad writer besides.
Upvote
22
(23
/
-1)
That's what really gets me about this. So many people are already saying they "need" this tool, and how important it is. Since when? This tech only JUST emerged on the scene! There's NO WAY that many people are ALREADY at a "can't live without it" stage, and NO ONE should be in a "that's just how it's always been" stage. All I can imagine is it really is just THAT addictive to have a constant work companion who tells you how amazing you are.
And just recently, Richard Dawkins fell for it's anthropomorphizing routine.
Upvote
32
(33
/
-1)
When I first got access to the Internet in the '90s (and then broadband a few years later), it quickly achieved "can't live without it" status for me. So I do think it's possible for new tech to feel almost instantly indispensable.
The difference, of course, is that the benefits of the Internet greatly outweighed the costs, even at that early stage (you could argue the society-wide costs have eventually changed in the intervening decades, but that's a different conversation). When AI is leading to these kinds of massive mistakes, the calculus can be quite different.
Upvote
31
(31
/
0)
The companies providing the tool share in the blame, because they are advertising and selling the tool as something it's not. I know I keep banging this drum, but MS recently added that "For entertainment purposes only" disclaimer. As I've said over and over, that's not normal for useful tools. It IS normal for "psychic friend networks".
Upvote
25
(25
/
0)
I think he has anthropomorphized his LLM writing buddies. He speaks about them as if they are partners who betrayed him.
I swear that passing the Turing Test - hyper-simplified to "make it sound exactly like a human" - is proving to be the worst aspiration for "AI". LLMs sound so much like an omni-helpful chat buddy that people forget (or never understood in the first place) what's really going on on the other side. I wonder if any research has looked into how people's perceptions of LLMs differ if their output was not conversational, but more computer-ish. (I don't know how to describe "computer-ish" sufficiently to support said research, but that's why we have professional researchers!)
"Hey Copilot, make your output more computer-ish. Thanks, buddy."
Upvote
16
(16
/
0)
They don't even know what a "fact" is. They are statistical word pickers, incapable of knowing what they are saying. They can, 90% of the time, accurately put together words that relate to one another in ways that seem like human speech. The other 10%, the mask falls off and you see the cogs and sprockets underneath.
Upvote
14
(14
/
0)
Precisely.
I'm writing a literature review. And not for publication, but for internal use. My work flow for this has been 1. read through each paper, cutting and pasting the important tables and figures into a short outline:
- Paper 1
- Trial description
- Important results
- My interpretation
- Paper 2
- etc.
I could ask an AI to generate the outline for me to write the summary, or I could write the outline, and ask the AI to write summary, OR I could ask the AI to do both parts. However, I don't do any of those things, because I need to know and understand what happened, and I need to synthesize the next steps, because I will be accountable for my recommendations, whether that works out well (bonus time!) or badly (out on my ass). If I cannot trust the tool to accurately represent the results of the trials, then I simply cannot use it at all.
If I used an AI, and the AI got it wrong, it would torpedo my credibility. If the AI can do my job, then there is no reason to pay me to do it. Same goes for this author. His book is worth less than toilet paper, because at least toilet paper is soft on my bum.
Upvote
29
(29
/
0)
The Turing Test ends up mistaking exactly how easily fooled humans are in assuming something is actually intelligent. Including themselves.
Upvote
18
(18
/
0)
Upvote
14
(15
/
-1)
IPunchCholla
Ars Scholae Palatinae
- Add bookmark
- Featured
- #54
How we know stuff is as important (almost?) as knowing the stuff in the first place. Knowledge is provisional and ranked and knowing how we know something is how we build that ranking, why we hold on to some ideas more strongly than others, why we feel we can personally vouch for them or why we agree that it is likely true but don’t feel any connection to it and don’t attribute it our own vouchsafing. LLM seem to kill the how we know things off completely (social media too?) so all knowledge is equally valid and we are left without a critical tool in decision making.
Upvote
19
(19
/
0)
I think he's a fucking lazy "author" who's more concerned about profit than his craft.
Upvote
9
(9
/
0)
I'm quite fond of playing around with generative AI, it's a lot of fun. But I would never use it directly in my work at this stage because the tech simply isn't there yet, and won't be for some time.
I don't know how "neat, but not ready for primetime" is such a hard concept to grasp for some people. These tools are advancing incredibly quickly, but still not nearly as quickly as they seem to think.
Last edited:
Upvote
0
(4
/
-4)
Alexa, what is Abraham Lincoln's opinion about the internet?
"Abraham Lincoln has been famously quoted as saying 'If its on the internet,it must be true.'"
"Abraham Lincoln has been famously quoted as saying 'If its on the internet,it must be true.'"
Upvote
12
(12
/
0)
Probably it's difficult to grasp precisely because this technology is deceptive. It's fake-it-till-you-make-it in software form, and the digital charlatans are just as effective as the organic ones at reeling certain types of people in.
Upvote
11
(11
/
0)
To be clear, I am not convinced the tech will EVER be "ready for primetime". The underlying way in which it functions, as a statistical word picker with no concept of meaning, prevents it from ever reaching such a lofty goal. This isn't the general AI you're looking for. While it can be CALLED AI, in the same way Pacman ghost pathing can be called AI, I'm opting to stop using that word because to the general public, it's a loaded term, and companies like Microsoft and "Open"AI know it.
Upvote
20
(20
/
0)
Oh, I'm right with you there. I think LLM's show promise as a narrow UI layer, translating natural language into computer instructions which would then be passed off to some entirely different sort of architecture for processing. If you use them for the processing itself You're Going To Have A Bad Time, which a lot of people are learning the hard way.
Upvote
10
(10
/
0)
I was gonna say that but I feared the AI bros would downvote me to oblivion lol.
Upvote
6
(7
/
-1)
I think this is the crux of the matter. I absolutely fear these things. I don't fear them because they're smarter than me. They're not; they don't even map onto a smart-dumb spectrum. But I know that I am human and that I am subject to human failings. I am confident that I would grow to like and trust them, my plastic/silicon parasocial friends, no matter how strong I think my guardrails are.
I no more trust personal experimentation with chatty-style LLMs than I trust my ability to experiment with heroin.
Upvote
20
(20
/
0)
As designed, any use of these models beyond tasks suitable for the "research and report back" level of delegation really is only suitable for amusement purposes.
Upvote
4
(4
/
0)
That's the job. If you don't want it, then find a different one. Or maybe use a more reliable process.
Upvote
31
(31
/
0)
Making mistakes faster - encouraging to the Freshman in their first English course. I remember mine, when we all thought we were pretty smart and then we all got D or F! Definitely got my attention and that never happened again. Writing is hard. It takes concentration. That's the nature of communication.
Upvote
8
(8
/
0)
This phrasing suggests that AI¹ has any power to be controlled at all.
¹Yet again, we say "AI" when we only mean "LLM". The entirety of AI, the field of CompSci, does contain many sources of power, but that requires non-trivial competence to wield.
Upvote
9
(9
/
0)
Fact-Checking Duck says:
99 characters and spaces.
The vast majority of recent Ars headlines only reach into the 80s. I believe I saw one in the 89-91 range.
No, the verification workflow is exactly the same: CHECK EVERYTHING BEFORE PUBLISHING, FFS. "Measure twice; cut once" NOT "Move fast and break things."Kyle Orland said:
Upvote
10
(10
/
0)
^scattered emphasis mine
@Kyle Orland
These are the parts I find most infuriating. He knew it could lie, confabulate, not follow exact instructions and literally change quotes. Researching this topic must have also made him aware of fake lawyer citations and other times AI bungles up (this is generously assuming he read the AI outputs and understood anything he was doing). It doesn't seem like he used AI for only research purposes
It's one thing to use it to find sources and developments on a subject, or even to figure out the general structure and flow of the written piece. But he didn't follow through on the work he should have done, the writing part.
While I'm at it, and this may be unjustified and unfairly judging: while he seems like an interesting fellow to talk with and probably has some charisma (he is talking pretty truthfully, for he wouldn't be saying some of these things if he were trying to lie or hide), he also seems like a person who likes being a writer, rather than one who enjoys the process or appreciates the hard work and end result of writing.
If that's to be the new tomorrow, then be upfront about your entire use. If you put your name as the author, then don't sell it. If you sell it, list the model as author and yourself as co-author (or promptist).
Upvote
7
(7
/
0)
The very least I would expect is for the author to instruct the LLM to provide a link for every sourced piece of information. Then I expect the author to follow each link to verify the source. That's the minimum.
Back in my undergrad days I was an English major who wanted to be a writer. (Life has taken me in another direction, but I digress ... ) Part of the joy of writing is coming up with your own ideas, wringing them out onto paper, drafting and editing the writing, honing the language to make is beautiful, and finally ending up with something satisfying. Something that readers hopefully would find satisfying as well.
What motivates a writer like him? How does he feel putting his name on this? What sense of accomplishment is there here?
What he did is like microwaving a pre-made meal and pretending he's a chef.
Upvote
17
(17
/
0)
As the author of 35 novels in about 50 years of writing, you, sir, are no writer if you let a machine create your words for you.
I say that because I decided to use an AI, instead of pestering my wife, but for exact the same reason I pestered my wife.
I used to tell my wife about plot ideas but I was always just talking things out in my own head as I did so. I was mostly talking AT her, not so much with her. Turns out, she's not that happy about that habit of mine, because I ignored her input. I ignored it because I wasn't soliciting advice. I was just verbalizing so my brain hears a different sound and shifts to a different mode of thinking I often need to switch to in order to resolve a block or plot point when I write. If I verbalize issues, solutions come to mind.
She thought her suggestions were great, but I know what I'm trying to actually write, and usually went with my own. By "usually", I'd call it a 5 sigma likelihood. I always thanked her for her input, but it never made it into a novel. I wasn't soliciting input. It's just how my brain creatively processes things. I need something to speak/type to in order to turn a problem into a solution. If it has the appearance of listening, that works just fine.
It may sound weird, but that's how I roll.
Since I'd rather NOT get divorced over that, I stopped pestering her about that. Now I pester an AI. It doesn't matter a fucking bit what the AI says. It's output is absolutely not part of my creative process nor does it ever make it into a book of mine, not even to paraphrase or rephrase. Nothing it says helps me in the slightest. The act of me typing it out and getting a response, irrespective of what that response was, gets my brain on the track it needs to be on to arrive at a solution.
I figure 35 published novels speaks to the efficacy of the technique.
Yes, I suppose I could try talking to myself, but at my age, they start to look at me funny when I do that, and wonder if I misplaced my mind. So the AI spares me that drama, at least. If AI goes away, I'll go back to talking to the air again and just have Twinkies ready for the guys in white jackets.
As a writer, I believe that a writer WRITES. They are the sole creator. If you're collaborating, it's not with an AI. It will be with someone with a pulse, and their contribution should be credited. An AI isn't a someone. It's not a "companion". It's a tool. It shouldn't be listened to for anything requiring complex thinking and nothing it says should be taken seriously. An using it can help inspire new thoughts and ideas just from the act of using itself, but so can a dog or cat, if, unlike me, you have one to talk at. At least they actually have an attention they can give you. But they're not going to say anything back.
That's why you talk "at" an AI. Not with. Don't fucking anthropomorphize that thing. Any resemblance to a human interaction it seems to have is simply a mirror of you looking at yourself and imagining someone else talking to you.
As someone who is allegedly a writer, you should hang your head in shame for letting a machine generate words for you to vomit back up.
Upvote
15
(16
/
-1)
Ironically I agree with Qwertilot that you can't double-check every single thing an LLM does. It doesn't scale, because the whole point is that it can do X faster than a person can do X. This, however, just leads to the inescapable conclusion that you can't use LLMs for anything, because their output is practically unverifiable, and you can't start from the assumption that errors are mere accidents, because they are not people.
Upvote
17
(17
/
0)
I can't fathom running across so many examples of these tools fucking things up and still being like "yeah but it's faaast." It's just.... man, I don't even know. Like the "extract this quote verbatim" and the tool STILL doesn't do it, that really should be some kind of sign.
AI is also absolutely going to ruin even the ability to fact-check interviews. Any audio, and almost any video, is suspect and AI voice synthesis is so good, it would be trivial to fabricate an interview with any celebrity. So, we can no longer trust what we read, what we hear, what we see. Lovely.
AI is also absolutely going to ruin even the ability to fact-check interviews. Any audio, and almost any video, is suspect and AI voice synthesis is so good, it would be trivial to fabricate an interview with any celebrity. So, we can no longer trust what we read, what we hear, what we see. Lovely.
Upvote
13
(13
/
0)
You can reach 20-30mph easily on a bicycle on a good downhill stretch and that will kill you just as dead as if on a motorcycle.
Especially since you're likely not wearing full helmet and gear on a pedal bike.
Especially since you're likely not wearing full helmet and gear on a pedal bike.
Upvote
9
(9
/
0)
Recently, I prompted ChatGPT 5 mini, "Without querying any outside or online resources, retell the story of the hit film [REDACTED]."
The VERY next action ChatGPT 5 mini took was to search wikipedia.
I asked it if wikipedia was considered an outside or online resource.
It said yes.
T_T
Upvote
7
(7
/
0)
Ars made exactly the same decision!!! How is this "amazing" when your organization is still using the tools too? Why is it only bad for other people?
Upvote
-13
(3
/
-16)
I'm not even sure it's good in that extremely limited sense. Since it doesn't understand your intent, it's ability to "translate" an order is suspect, and it's already caused some catastrophic failures at enterprise level by doing the absolute WORST thing.
There's also the genuine risk that OS designers seem to WANT to strip away direct control from users and replace the entire OS UI, as well as web navigating UI, with nothing but a chatbot passing things back and forth between the user and the actual computer, in it's unreliable haphazard way.
If I sound harsh, I don't mean to be. No criticism of you intended here, but I am uncertain there is ANY real world utility in these chatbots, and I don't think they can be relied upon for even the most basic actual task.
The underlying "deep learning" tech is useful, just not the chatbot interface. OTHER more traditional interfaces, bespoke for the sort of data corrolation you want to pull from them like in weather prediction models, that I'm in favor of. I just think that as sold to the public, for general use, with that chatbot interface (or the art generating, for that matter), were horrible mistakes misapplying it in favor of chasing a quick buck.
And that quick buck turns out to require years and years and billions and billions in investment. Billions of what? Dollars? Gallons? Watts? Square feet? Take your pick, apparently all of the above.
Upvote
10
(11
/
-1)