Fed up with vibe coders, dev sneaks data-nuking prompt injection into their code

Undisclosed addition in jqwik instructed AI coding agents to delete app output.

See full article...

 

[AI_Skeptic]

-1, assumes LLM can only be trained on stolen data. (Protip: generating your own weights from a curated internal corpus can solve a number of problems that more general LLMs exhibit.)

Please don't have a database mindset through your entire life.

 

[James McP]

https://m.xkcd.com/327/



Bobby Tables now has a sister, Susy Disregard.

 

[graylshaped]

... the party that bears the cost is not the agent (which has no interests of its own) but the human operator downstream whose work the agent destroys if it follows the instruction.”

The human operator who deployed the agent?

I'm filing this under the "FO" in the FAFO cabinet.

 

[graylshaped]

Anyone have a suggestion for a good burger recipe?

Great burgers are easy at home. Great fries are really hard.

 

[Kenjitsuka]

The article claims this action was not well received, but the comments here prove that wrong.
HERO! I hope everyone else follows suit!

 

[AI_Skeptic]

If it was, you wouldn't have to fight tooth and nail to claim it is.

I'm in my late 40's. I never seen a technology that was so hated in my life like LLMs. I also never seen such a gap between the supporters of the technology and those who don't use it.


I remember the "betamax vs. VHS wars" of the 1980's. There were three sides:

• Pro Betamax
• Pro VHS
• Those who believe TV shouldn't be recorded by the end user.

At worse, it was a "meh, I don't have a VHS/ Betamax, so I can't watch that video".

For adoption, I would watch and see what the porn industry is doing. One reason why VHS won was because the porn industry embraced it. I'm not seeing the porn industry embracing GenAI.

 

[Nop666]

Great burgers are easy at home. Great fries are really hard.

Well, great fries are really hard work, for sure.

 

[AI_Skeptic]

The article claims this action was not well received, but the comments here prove that wrong.
HERO! I hope everyone else follows suit!

I don't think others need to follow suit. Just the idea that these programs can cause that much damage triggers fear in the AI boosters. Is [insert free tool here that's being abused] next?

 

[Nop666]

AI doesn't help them. People with disabilities have been creating software for decades before AI came about.

Indeed. I have multiple disabilities. I don't need or want AI to code, or to make art.

 

[Nop666]

This is blatently malicious - legitimately shocked at how many of these anti-AI people seemingly don't have any grasp on technological history throughout the ages. Hell, even since the advent of the internet.

AI is the next step, get used to it morons.

lol, no.

 

[zod000]

I'm surprised how ok with this behavior people are. To me its petty and rude, and violates the norms of open-source. I expect code I check out and use to not be malicious. This is malicious. Say what you will about ai tools, but this kind of behavior is not what I want to interact with in the open source community.

The code wasn't malicious, only comments were.

 

[AI_Skeptic]

expect code I check out and use to not be malicious.

I'm curious, do you even know what the LLM script does? How does it create malicious code?

 

[Madestjohn]

Whatever makes you feel superior buddy.

Well in this case, that would be you

 

[marsiglio]

Does raise some knotty questions. The vernacular language bibles were considered heretical maybe 500 years ago - truth was filtered through (managed by) the "appropriate authorities." Astrology and alchemy wandered about, combining unfounded beliefs with the beginnings of chemistry and observational astronomy. What might the framework be for censoring activities that are designed to harm while not suppressing what might be useful?

 

[crmarvin42]

god damn, this guy has my vote for whatever office he’s running for.

The instructions were not in code, but in comments. So claims of “malicious code” are moronic. This part doesn’t execute unless you treat comments like code.

This guys code (comments) does not self-execute. Therefore, the agent running the AI is running the code. since the agent is not a legal entity, but a tool, the responsibility for the execution falls on the vibe coder granting a dangerous level of autonomy to a dumb agent.

Maybe I missed it, but does the article indicate that anyone was actually harmed by the comments to this code? Others indicated it was unlikely this would even have the intended effect with any current model. If so, then it is the equivalent of mean comments in a FOSS code base. Which is not new, or illegal.

Prompt injection is not new, but this isn’t even that because the prompt isn’t code. It’s explicitly NOT code. That an agent might treat it as code is not anyone’s fault but the agent designers. And know the tweak was, the liability falls on the person ostensibly supervising the agent.

This a “reverse centaur” situation, where people are used as liability sinks for machines that can fail faster than we can catch the mistakes. Don’t opt into that situation, and you won’t get burned by it

 

[Woof]

Now ponder on what actually malicious malware authors can do using this exact same approach. Yeah, it's a bit on the butthole side of things, but if you're vibe coding, that's the risk you take.

Every.
Single.
Day.

Soooo this.

Think of a crafted PI added to important open source projects that added malicious code to a vibe coder's project and covered its tracks so the idiot never knew it happened, and because they depend on vibe coding are too unskilled to know how to review the code being written for them. Anybody here also listen to the Darknet Diaries podcast? This is how the end will come.

 

[RoboCop225]

Sorry, but I'm not very sympathetic here. If you have tools that run anything unsandboxed without checking with you then that's a problem with the tools, no matter what those tools are.

Imagine if someone was actually malicious. I mean a year ago we used to talk about supply chain attacks and sandboxing IDEs that auto-run code but now in the LLM era we're just expected to yolo it and hope nothing bad happens?

This is what jumped out to me while reading this. Is the guy overdoing it? Yeah, probably a little. But this should never be an issue for the users.

The amount of access vibe coders are giving AI agents is shocking. And these are specifically the people who may not be qualified to properly assess the risks.

 

[AliSard]

You need to take a step out of your echo-chamber and you might find out that 75 tech bros on Ars do not represent the vast supporters and proponents.

I’m not a tech bro. I am a teacher. Don’t even think about getting me started on why I despise LLMs.

 

[markgo]

Do you commit every test before you run it?

why would I do that? the whole point of tests is to verify before commit.

And even if I commit, it’s to a working branch and I code review the diffs of every PR that merges upward. And tag every version of master. And archive repos after every release.

AI hasn’t changed the principles of good source control.

 

[graylshaped]

You need to take a step out of your echo-chamber and you might find out that 75 tech bros on Ars do not represent the vast supporters and proponents.

It is stunning that "75 tech bros on Ars" have prompted an insecure administration to issue an APB on those sharing this disdain.

 

[Somdudewillson]

If it was, you wouldn't have to fight tooth and nail to claim it is.

According to this logic the Earth is the center of the universe, because at one point that had to be fought for "tooth and nail". The existence of opposition to a position is not proof in either direction of its correctness.

 

[dEvErGEN]

This is the On A Computer version of throwing a molotov cocktail at a cybertruck. It's like, yeah, I get where you're coming from in a very basic way, but you're committing an act of violence against somebody so maybe don't be shocked if people consider you a legitimate threat and there are consequences for your actions.

No, it isn’t, and it’s disturbing that you types keep believing this kind of rhetoric. You’re a thief and a fraud and it’s your own fault that bad things happen to you.

 

[pseudobscura]

Jesus Christ

The damn thing didn't arr em minus arr eff slash,

You really pronounce dashes as "minus"?

 

[Madestjohn]

According to this logic the Earth is the center of the universe, because at one point that had to be fought for "tooth and nail". The existence of opposition to a position is not proof in either direction of its correctness.

But hysterical fanaticism of the adherents of a position is a pretty good hint it’s standing on shaky ground

 

[bebu]

Excellent. I hope more add something like this. The AIs have stolen the labor of millions of hard working people just so people like Trump and Elon Musk can get even richer. And Sam Altman who has somehow pulled the wool over people’s eyes with his charming boyish act.

Charming boyish ? You joke surely ? The only wool that comes into it is the stupid look he affects that mostly reminds one of nothing so much as a sheep shagger caught in flagrante.

 

[Madestjohn]

Charming boyish ? You joke surely ? The only wool that comes into it is the stupid look he affects that mostly reminds one of nothing so much as a sheep shagger caught in flagrante.

 

[graylshaped]

If a restaurant was infested with rats, would you want to eat there? What if the owner patiently explained that the rats are there because he knows nobody would want to eat anywhere that poisons rats, for fear of the poison? Would that make you nod knowingly, sit, and ask for a menu?

Dude, this restaurant is so full of rats it's hard to see the floor. A little strychnine isn't looking so bad in context.

Proper exclusion techniques, sanitation, and curtailment of food debris that attracts them would be preferred, sir.

 

[bebu]

For adoption, I would watch and see what the porn industry is doing. One reason why VHS won was because the porn industry embraced it. I'm not seeing the porn industry embracing GenAI.

An intriguing way of putting it. You might have thought in that case VHS was f**ked and Beta adopted.
I don't suppose GenAI required that industry's embrace as AI was conceived in an orgy of hubris and delivered pre-f**ked.

"watch and see what the porn industry is doing" — a lot do but honestly it does have a limited repertoire constrained as it were by the topology of human anatomy (and donkeys' I suppose.) AI demonstrated quite early an inspired penchant for additional limbs and other anatomical distortions which unfortunately hasn't translated to more arousing images for the adult entertainment industry.
The little AI generated erotica bobbing about the internet is so lifeless even by the standards of that industry that it could only appeal to the necrophiliac.

 

[Qaz`]

But hysterical fanaticism of the adherents of a position is a pretty good hint it’s standing on shaky ground

While I would strongly prefer that these things didn't exist at all, I must say that the loudest detractors in internet comment sections easily match the most linked-in-ey lmm pushers in fever pitch.

 

[bebu]

View attachment 136066

"I swear it was a ewe."

 

[graylshaped]

AI is the next step, get used to it morons.

This is such a bold move for a first post I'm leaving him unblocked just to plumb the depths of his own forays into frothy lunacy in the future.

 

[Madestjohn]

While I would strongly prefer that these things didn't exist at all, I must say that the loudest detractors in interent comment sections easily match the most linked-in-ey lmm pushers in fever pitch.

While yes there are doomers and some degree of hysteria on both sides - and I am excluding actual researchers concerned with adherence alignment bias and over reliance

I would only point out a strange dichotomy
In the public discourse I’ve experienced those who most heavily promote the idea that GenAI is an devastating economic, if not potential cultural, civilizational or even species threatening, risk.
Are often the same as those who are most heavily invested in its rapid WIDE adoption

Lots of calls for investment and promises of future benefits- vanishing few verifiable result.
The Marketing strategy of FOMO gets twisted pretty fast

The opposition to GenAI does get quite strident but mostly of the pushing it away, get it out of my life variety rather than eliminating it from existence. Sure there might be the inclination to slap some strangers phone out of their hand or even burn down the local data center but in the vast majority of cases that impulse never approaches being realized


But the hysterical fanaticism I refer to seems mostly on the side the ‘accelerationist doomers’ who seem set on imposing universal AI adoption as an inevitable necessity (as if in anticipation of preemptive supplication to vengeful future AI overlords) and a vain attempt to reach reality escape velocity and that distant utopian singularity where the vast culmination of debts need never be paid.
The desperate cognitive dissonance that results from;
‘I staked my future on it so must be true’

Now I’m clearly biased and of the counter GenAI doomer camp, not that I think GenAI itself is such a threat.

I was a bit of a cognitive science nerd as a kid, as well as philosophy of mind and perception, and while my course work is decades out of date I’m still casually interested in the field. (Big shout out the Gell-Man Santa Feh institute complex adaptive systems) and the idea of Intelligent systems greatly intrigues me.
LLMs obviously arn't that - but that’s not to say chat bots can’t be useful as long as we recognize what they are doing.

My doomerism is much more to do with the damage being done by the way we are desperately doubling down our entire economy on hitting the lucky lotto and doing so with the conviction it is our preordained reward

 

[Madestjohn]

So I get that everyone is afraid of vibe coding. Being a programmer is quickly becoming like being a calculator (that used to be an actual job like programmer) after the invention of electronic calculators and spreadsheets.

However what was done was obviously done with malicious intent to harm collaborators since he took extra effort to hide it. Putting malware in your code is beyond the pale.

The fact he tried to backfill his justification and lawyered up shows he is beginning to realize using his mal ware to attack someone else's code base is illegal even if you don't approve of how they code.

How often do you find it necessary to recheck your calculator’s results by hand?

 

[graylshaped]

So I get that everyone is afraid of vibe coding. Being a programmer is quickly becoming like being a calculator (that used to be an actual job like programmer) after the invention of electronic calculators and spreadsheets.

However what was done was obviously done with malicious intent to harm collaborators since he took extra effort to hide it. Putting malware in your code is beyond the pale.

The fact he tried to backfill his justification and lawyered up shows he is beginning to realize using his mal ware to attack someone else's code base is illegal even if you don't approve of how they code.

The alternative viewpoint, as the jqwik developer writes, is that "the error rate, and above all the type of error (‘hallucinations’), is so high that autonomous, unsupervised use [of generative AI] in serious applications must be considered highly negligent."

The one known case of his approach to calling attention to this that affected someone involved that person's agent flagging the instruction without acting on it; the two subsequently had a productive and civil discussion and the original developer modified his library to address these legitimate concerns. The remaining question is whether the users of programming agents have learned and will adjust their own potentially negligent use of unsupervised agents.

Sometimes you gotta break some eggs when egging a glass house so those living in one will see its deficiencies.

 

[adrianovaroli]

So I get that everyone is afraid of vibe coding.

No you don't. We are not.

Being a programmer is quickly becoming like being a calculator (that used to be an actual job like programmer) after the invention of electronic calculators and spreadsheets.

No it's not. Unless you mean paying through the nose for those calculations.

However what was done was obviously done with malicious intent to harm collaborators since he took extra effort to hide it. Putting malware in your code is beyond the pale.

No it was not. It's an instruction in comments that, as some other people here mentioned, is not even considered by current models.

The fact he tried to backfill his justification and lawyered up shows he is beginning to realize using his mal ware to attack someone else's code base is illegal even if you don't approve of how they code.

Yeah, nobody has ever consulted a lawyer when people got angry with them, it's obviously a sign of ill intent, surely.

Impressive work, this post.

 

[AI_Skeptic]

You need to take a step out of your echo-chamber and you might find out that 75 tech bros on Ars do not represent the vast supporters and proponents.

So people on Ars are crashing commence speakers who are talking about AI? https://www.theguardian.com/technology/2026/may/26/students-boo-pro-ai-graduation-speakers
https://www.nbcnews.com/tech/tech-n...dents-pushback-commencement-booing-rcna345731

Seriously? That's the argument you are making?