Frustrated software developer believed AI-generated message came from human support rep.
See full article...
See full article...
Cursor AI support bot invents fake policy and triggers user uproar
- Thread starter JournalBot
- Start date
Vibe coders using an LLM driven workflow given wrong information by an LLM chatbot that they didn't error-check.
Completely on-brand for vibe coders. (Full disclosure I use Github Copilot for low-stakes refactors, line autocompletes, and trivial API lookups. But I error check statically damnit.)
Completely on-brand for vibe coders. (Full disclosure I use Github Copilot for low-stakes refactors, line autocompletes, and trivial API lookups. But I error check statically damnit.)
Upvote
273
(278
/
-5)
If I subscribed, I'd cancel on principle. Shady business practices should not be rewarded.
Upvote
147
(148
/
-1)
LinkedIn is flooded with terrible takes about how we like, won't need version control anymore, mannnn, like GitHub you know? Because our code will just like be the prompts, and the prompts are your code, mannn.
I ran out of characters in the comment field well before I ran out of ways to point out how dumb that was.
Upvote
238
(238
/
0)
Back in my day when a product continually failed to perform within expectations, cost a company money and could not be repaired, instead of inventing cute little terms for why the machiney-weeny did an oopsy-whoopsy we rightly determined it was a bad product and got rid of it while suing someone.
Upvote
248
(250
/
-2)
Narrator: Though the reason for the cancellations had changed, the cancellations continued because coders want humans to help them deal with issues caused by AI-generated vibe coding subs.
So, question: If you can continue to use Cursor on multiple machines, why didn't that particular coder experience the continuity of doing that across multiple machines?
Sounds like they still have a problem, and no current fix for it.
Upvote
71
(79
/
-8)
It's only a matter of time before one of these dumb things gets somebody killed.
Upvote
144
(145
/
-1)
While I appreciate Benj's diligent work on the AI beat, I remain disappointed at the uncritical acceptance and parroting of the anthropomorphizing language the industry uses to build its hype train, i.e. "confabulation", "hallucination" etc. Why not "error", "random output", "bad data", or if we want to be technical, "bullshit"?
Upvote
328
(347
/
-19)
My engineering director at work recently asked me how I can use AI to help me speed up my development, like using it to do cross-platform development between android and iOS and web, without knowing the other platforms or languages. We already don't have any QA team, and now they're prodding more and more vibe coding. I didn't think much of him before this meeting (too much hearing himself talk, fatuous questions), but now any respect I had is gone.
Upvote
195
(196
/
-1)
I love the response from the chatbot. “We limit a given session to a single device as a core security principle. Anyway, you can pay more for the functionality”
Upvote
131
(131
/
0)
- Add bookmark
- Featured
- #13
An aspect that seems to be overlooked isn't just that the users believed Sam to be a real person, but that the first level of technical support is presenting falsehoods. I don't care if I am getting information from a person or from an AI. I care that I am getting valid information. The addition of the phrase "We use AI-assisted responses as the first filter for email support" does not eliminate the actual problem.
Upvote
250
(251
/
-1)
FTA - there was a glitch in their session handling. They fixed… the glitch. The rest will just work itself out.
Upvote
52
(53
/
-1)
And what do you think the model you're vibe coding with is trained on…?
Upvote
162
(164
/
-2)
So question on that. How would they of checked that the support email response was valid and from a person?
Upvote
33
(39
/
-6)
So. The real problem here is with all Ai usage. It's a blackbox that end users just get screwed by. The company didn't disclose the response as Ai upfront. Users had no way to reach out to a real person.
The reddit side of locking the thread and deleting the post also is simply wrong and should not of happened.
The reddit side of locking the thread and deleting the post also is simply wrong and should not of happened.
Upvote
119
(123
/
-4)
No doubt this would absolutely be frustrating, but as someone working in a Support capacity I have to admit it's a little validating to know that A.I. still can't do my job.
Upvote
117
(117
/
0)
I haven't checked, but my guess is yes. He seems to get by on schmoozing and has the same offputting aura that I get from MBAs
Upvote
57
(57
/
0)
The mistake of it is bad.
The mods on the reddit sub deleting it is even worse. Cursor employees are mods in the sub. There has been no announcement if it is was a cursor employee or regular mod that deleted it. There is no mention of it on the pinned topics or in any other location.
I also don’t see that the post would have broken any rules of the sub or any reason it would have been deleted in the first place.
If cursor wants to keep trust they really need to explain why the post was deleted as well as make clear that in the future that will not happen if they can get get mods to agree or if the owners of the sub who I don’t think are cursor employees disagree have their own employees leave as mods and make clear the sub is not officially supported by Cursor.
The mods on the reddit sub deleting it is even worse. Cursor employees are mods in the sub. There has been no announcement if it is was a cursor employee or regular mod that deleted it. There is no mention of it on the pinned topics or in any other location.
I also don’t see that the post would have broken any rules of the sub or any reason it would have been deleted in the first place.
If cursor wants to keep trust they really need to explain why the post was deleted as well as make clear that in the future that will not happen if they can get get mods to agree or if the owners of the sub who I don’t think are cursor employees disagree have their own employees leave as mods and make clear the sub is not officially supported by Cursor.
Upvote
185
(185
/
0)
Hey, I just wanted to let you know that it’s “should have” and “would have.” The increasingly common error “should of” stems from the spoken version of the contraction, “should’ve”, “could’ve”, “would’ve.”
Cheers!
Upvote
169
(177
/
-8)
David Mayer
Ars Centurion
The question must be asked. How much money was saved by using AI and how much was lost by using AI?
Upvote
78
(80
/
-2)
I'm reminded of the old Chappelle's Show PopCopy sketch:
"If a customer brings in a computer disk, look at it and tell them it's the wrong format. If they use apple, tell them we're PC. If they use PC tell them we're apple. And If they have both, tell them we use linux. And if they got that, tell them the computers are down."
Because an ai agent will lie to customers with absolute confidence. It's not malicious, but it also doesn't care. It's not capable of caring.
From a business standpoint, companies rolling out ai agents should consider this as a risk: what if a human doing this job chose to lie, with absolute confidence, to the customer? What damage would occur? For a human you have countermeasures, you can train them or hire better humans, treat them better and pay them more. Or in the worst case you could hold them liable if they damage the company.
If my ai agent decides to say, break the law or damage the company there is no scapegoat. I "hired" an agent who will lie like a sociopath and also doesn't care.
"If a customer brings in a computer disk, look at it and tell them it's the wrong format. If they use apple, tell them we're PC. If they use PC tell them we're apple. And If they have both, tell them we use linux. And if they got that, tell them the computers are down."
Because an ai agent will lie to customers with absolute confidence. It's not malicious, but it also doesn't care. It's not capable of caring.
From a business standpoint, companies rolling out ai agents should consider this as a risk: what if a human doing this job chose to lie, with absolute confidence, to the customer? What damage would occur? For a human you have countermeasures, you can train them or hire better humans, treat them better and pay them more. Or in the worst case you could hold them liable if they damage the company.
If my ai agent decides to say, break the law or damage the company there is no scapegoat. I "hired" an agent who will lie like a sociopath and also doesn't care.
Upvote
110
(111
/
-1)
Upvote
12
(13
/
-1)
Upvote
160
(163
/
-3)
Upvote
34
(34
/
0)
"Oops. Our bot screwed up"
"But youre going to stop using ai for customer support then, right"?
"..."
"But youre going to stop using ai for customer support then, right"?
"..."
Upvote
76
(77
/
-1)
"AI models invent plausible-sounding but false information" is something I have in common with AI whenever my wife asks me something about her computer.
Upvote
46
(48
/
-2)
Upvote
17
(19
/
-2)
Bumbling Baboon
Seniorius Lurkius
Didn't these things already cause several deaths from suicide and such?
Upvote
50
(50
/
0)
"the chatbot is a separate legal entity that is responsible for its own actions."
This is the saddest aspect of the article. Reminds me of when Disney defended their abhorrent actions in the death in one of their park's restaurants because of a Disney + subscription T&C. Optics are really bad.
This is the saddest aspect of the article. Reminds me of when Disney defended their abhorrent actions in the death in one of their park's restaurants because of a Disney + subscription T&C. Optics are really bad.
Upvote
102
(102
/
0)
If a human tech support made up a fake policy, lying to the customer and embarrassing the company, he would be fired.
Upvote
80
(80
/
0)
"made the policy up" "invented"
These things all imply intent. There is no intent in an LLM.
Upvote
22
(32
/
-10)
How do we know that was the chatbot? Maybe the email was from a person, and the later post "correcting" the email was the actual chatbot?
Upvote
19
(19
/
0)
Unfortunately, this doesn't seem to be the most important criteria for management in too many instances.
Don't worry. In an attempt to solve the problem, they'll spend greater and greater sums of money, which is similar to your idea.
Good bot.Hey, I just wanted to let you know that it’s “should have” and “would have.” The increasingly common error “should of” stems from the spoken version of the contraction, “should’ve”, “could’ve”, “would’ve.”
Cheers!
Upvote
26
(26
/
0)
I’ve noticed AWS is using AI for first level support as well. Your first response on every reply is basically a summary of what you said and a promise to look into it, which I guess makes it feel like you’re being listened to and taken care of. It is a bit creepy how it is presented as though someone has taken the time to understand your issue, but in reality things have gone nowhere, just buys them time.
I guess if the summary is wrong it gives you a chance to clarify before a human gets involved.
I guess if the summary is wrong it gives you a chance to clarify before a human gets involved.
Upvote
51
(51
/
0)
Making up fake policies on the spot is par for the course for a lot of tech support, especially low-cost, low-training offshore outfits.
Upvote
12
(17
/
-5)