Opinion: As software power tools, AI agents may make people busier than ever before.
See full article...
See full article...
Sounds like there’s a good chance of that.I wonder whether there is any chance that ChatGPT is reporting my results. Sort of freaking me out.
I wish I knew.
I read that in my head in a David Attenborough voice...“Meanwhile, the poor Python, by effectively removing all barriers to communication between LLM companies and hobby code scraped from Stack Overflow, has caused more and sloppier programming than anything else in the history of creation.”
In short, curation and exclusivity are necessary which means when demand is low there are more losers.Unfortunately, your last point is likely the most important - you can make things too fast.
The fact is, the coding industry is already suffering for that, as is the media landscape. It's already very inexpensive to make things quickly, and as any good economist will tell you, value is a matter of supply vs demand, and AI will vastly increase supply without doing anything to increase demand - the value of all the products so created will drop, precipitously.
If software was very expensive, this would be a relief - but it isn't. It's already cheap, particularly in the entertainment field, and taking the cost trend too far is also bad for industries. It devalues everything they do, potentially to the point of collapse.
Right now in the game industry, there are a lot more games than anyone can ever play. Even a gamer that wants to indulge entirely in a highly specific niche will likely find more than they could ever play. AAA development is being shredded by forces that it cannot manage, as they struggle to make the best possible game to get players highly divided attention - but most of them fail, burning hundreds of millions of dollars in the process. At the other end Indy development is booming and creating a renaissance in gaming, but even in this idyllic consumer case most indy developers will certainly fail even if they make a very good game, because only a small number can ever bubble up into public view, even with a surprisingly egalitarian platform like Steam aiding discoverability.
This is all before AI has had a real impact on development, and unfortunately the results will likely be disastrous for the industry, with an appalling flood of new content that no-one will ever see. Amazon's self-publishing book section has already gone down in flames under an onslaught of low-effort low-quality AI books flooding out the efforts of any human author - but they aren't making money either, no-one is, and a sector that was growing and thriving is suddenly basically dead.
Like cheap-ass plastic toys and kitchen utensils, there can be such a thing as too cheap, and it can badly damage entire market sectors, destroying the ability of people to engage with that market as they are flooded with mass produced spam that makes discoverability of quality releases functionally impossible, and drives the value of even high quality releases close to zero - eventually consumers give up and go find someplace to spend their money where they can still find some tangible value for their money.
There have been attempts at this, with experienced programmers, not a whole spectrum of levels. They didn't show that the LLMs actually helped.Anecdotal results are entertaining, but that is mostly what I'm seeing across the board. I yearn for articles that would at least have two people of equal skill tackle similar problems over the same timeframe, one leveraging LLMs, the other not, and have the results judged by an independent third party. In an ideal world we'd have several groups of people at different skill levels all taking part for a better data set. This would help alleviate intrinsic biases humans have especially when interacting with LLMs that have been primed for sycophantic responses.
A buddy of mine had one of the agents build him a Syllabus for a class to teach him Rust. I looked over it. It wasn't great, but I'd say it wouldn't've been out of place in a high school class room.More and more as time goes on, I feel like I am the odd duck out here. I actually like to learn stuff, I want to spend the time and effort to grasp things myself instead of relying on LLMs, I'm not afraid of having to put some work into it.
I randomly decided to start learning Rust like two weeks ago or something and I've been programming something akin to Autohotkey as my learning project and I have used Gemini while at it, but I've only used it as a learning aid, not to design or write the code for me -- I pop in to ask if I can't quite figure out how to do something or if I am hitting an error I can't quite grasp and since I am not familiar with the win32 API, it has been useful in figuring out some of the functions I should use to achieve something, but it's very much just an aid for me to find the information I need faster. I could still get all this done even without it, it'd just take some more time for me to e.g. browse through the win32 API documentation.
And then I see the constant stream of Reddit-posts, YouTube-videos, various blog-posts and so on all just nilly-willy using LLMs without trying to learn even the tiniest bit themselves or putting in any more effort than just writing prompts. It's an fscking constant stream of shite and it annoys me. It makes me worried: I feel like people like me, people willing to learn, are a dying breed.
I found the opposite. I use it to do the easy 90% of the work, leaving the challenging 10% for me to finish.My biggest issue is that it has sucked the fun out of programming for me. I used to enjoy the problem solving and satisfaction from creating things myself.
The productivity boost is too large to ignore with AI, but the end result is I feel most of my work is just code review now.
I hear you, I feel the same way about people using Python for anything serious. BUT, using CE would have a distinct advantage (theoretically): hardware independence. Not that your boss would know hardware independence from a hole in the ground, of course.High level languages are all very well, but the amount of bloat is somewhat maddening for a microcontroller hardware guy like me. [...] The project manager was touting "time to market" and "I'll take that risk", and pushed us to use Windows CE instead of other options.
Absolute hogwash, they won't learn any of the fundamentals anymore! We're already seeing CS grads who don't know a pointer from Shinola, or don't know anything about data structures. It's a far cry from my days where a Data Structures course involved a lab coding everything from a linked list to a balanced B-Tree from scratch, and yes, with pointers; or a programming course where the introductory lab was "here's a recursive QuickSort, make it iterative". Those two courses caused about half of the students to leave; by year 2, EVERYONE could program. Now most CS graduates we see take a week to write 5 lines of Python. It's pathetic.Kids who are interested in learning programming at this moment in time, will have a huge advantage compared to the nerds from decades past. They will learn quicker. They will have so many more reps under their belt.
This. A zillion times this. It's the definition of being in "maintenance mode" from day one, which is clinically insane, and paralyzing.No, it creates instant, infinite technical debt and legacy code. There is no one to explain why certain choices were made which is a definition of legacy code:
"Nobody understands the code that the AI wrote, but at least the tests pass and it conforms to all the code standards!" "Cool, who wrote the tests?" "The same AI,." "Oh, who is checking coding standards?" "The same AI." Nobody is going to fully code-review the fourth blanket code review the new intern told AI to do, not if it's a 20,000 line change, and there are only 1 or 2 people left who actually understand what the code does.I suppose you could argue that if you have very strong confidence in tests then maybe it doesn't matter so much what the actual code is as long as the tests pass
I don't think any of the senior professional devs here are saying that. There will still be plenty of work -- most likely, there will be even more work.This is exactly why these tools will not result in significant loss of work for software developers.
Which makes it kind of hilarious to have the near-universal judgement of experienced programmers, essentially being "yeah, it's neat for hobby shit, but it is utterly useless for the things I do" be dismissed by people who -- at best -- have programming as a long-term hobby.There have been attempts at this, with experienced programmers, not a whole spectrum of levels. They didn't show that the LLMs actually helped.
But it's a difficult thing to really study. There isn't a true way to A/B someone doing the same task with and without a tool.
Oh, don't worry, that correction will come. The entire idea for programming LLMs is to get enough actual developers fired to have crucial businesses wholly dependent on the AI teat for developing new features, writing tests for it, doing the testing, and doing the deployment.There are the practical costs (currently decoupled from any pricing scheme),
And why would they give a fuck about any of that?the environmental costs (both to people's local environments in proximity to datacenters and larger global ones due to increased consumption of carbon-rich fuels),
And why would they give a fuck about any of that?and societal costs [all the horrific shit we already know, it's just too depressing to re-read]
Every time I've automated/controlled something with FTP, I've eventually replaced it withEarlier I decided there were two things I wanted to experiment with and learn more about and I combined them: learn Rust, and see what AI can do for me with code generation.
I had a test program I wanted that was kind of complicated: SFTP client and server, multiple worker threads passing jobs and synchronizing between each other, lots of configurability, a GUI interface, ... Definitely not the best choice for "my first Rust program", but I figured Claude could help me make it work. A test tool to automate tons of mundane repetitive tasks on the servers in our lab.
<snip>
rsync -e ssh.We're not currently hiring, but I will be hiring juniors in the future. But we're in growth mode. We want a bunch of people being productive, not the fewest number of bodies managing a fixed quantity of work. The more productive they are, the better.As a soon to be software engineer graduate turn wannabe junior developer, hearing established developers say "It helps me a lot but I feel bad for junior developers" increasingly sounds similar to, "Fuck you, got mine".
Umm, neat?Every time I've automated/controlled something with FTP, I've eventually replaced it withrsync -e ssh.
Although at some scale, stuff I've managed with rsync eventually got replaced with Configuration Management. I preferred Chef at the time, but I've heard good things about modern Ansible. Although I run everything as kubenetes pods now, so I just ship new container images and let AWS manage the nodes.
I guess the next question is how will we have next new big things? These tools can only regurgitate what they have been trained on. Therefore, how do you ever get new languages? If developers are no longer the providers of problem solutions, but only debuggers of LLM output, you don't have people working on the next new thing. Even if some passionate savant creates a next new thing, how would you ever promulgate it in a world where all content must come from the pre-existing slop machine?It's not like the "many" you are referring to have not seen attempts at major technological change that were smoke and mirrors. On the contrary, senior programmers have seen so many Next Big Things. We saw ActiveX, the Semantic Web, All Apps Will Be Java, All Apps Will Be Flash, All Apps Will Be Silverlight, blockchain, microservices, wearables, Segways, 3d TV, Web 2.0, IoT, Web3 (are we at the Web4 buzzword yet?), Big Data, NFTs... we saw them all.
The reality is that far, far more "new shiny bronze metals" are claimed to be the new disruptor but turn out to be a flash in the pan. We usually don't see the most disruptive tech trends until after it has already made the biggest changes.
say you weren't around at the beginning of the web without saying...The comment you're replying to is concerned less with AI coding as "additional abstraction" and referring to how untrained individuals can make apparent progress with tasking they're generally incompetent with.
It is not accurate to compare AI prompting to another layer of abstraction because unlike high level programming abstraction, AI prompting is neither a well documented or reliably interpreted methodology. Even programming languages that do not cleanly map to processor instructions, such as functional programming languages and weak-typing like Python has, have hard rules that form a specific, learnable system with predictable behavior.
It's something that a trained software engineer can sign-off on because they can directly study the known limitations of the abstract layer and how it works. This isn't true for AI generated code. You can't sign-off on the capabilities of AI generated code by looking at the prompt and performing simple tests on the compiled code.* A skilled software engineer who understands how a requirement needs to be met can prompt an AI, look at the output and verify the AI took the right approach without producing side effects; but an unskilled 'prompt-engineer' that's only familiar with coaxing answers out an AI can't because they won't know how a problem should be solved. For normal layers of abstraction that a software-engineer doesn't fully understand the implementation of, there is documentation to be read regarding the scalability, effects, and resultant mechanics of the high level layer. The 'prompt-engineer' does not have reliable writings and documentation to study, only syntax that accomplishes a task they don't actually understand.
* In theory, you can come up with and run a set of tests to guarantee that the code works. In practice, this requires an incredibly thorough understanding of your requirements, something the 'prompt-engineer' doesn't have.
** Throughout all of this, my definition of 'prompt-engineer' excludes everyone who actually has the expertise to perform the tasking at hand without the use of generative AI. For a skilled software engineer who knows what they're doing, an AI can just be another way of arriving at the syntax they need. The original post is referring to the hiring of unskilled AI-prompters as a cheap alternative to proper engineers.
1. grammar asking for code to be written is nondeterministicIn defense of Python, as an early adopter of it, the program logic and underlying math I use programming in it is the same as I’d use in C, Java, or any other language. The main advantage is that I have a bunch of data structures and libraries built-in rather than having to allocate the memory myself or use Boost libraries. It abstracts, but those abstractions are fully and clearly defined and I can delve into the details whenever necessary for debugging or improving performance. Or write the bottleneck code in something faster if necessary.
(There’s also a security advantage, in that not having to allocate memory myself stops me from screwing it up and giving someone access to the stack.)
The comparison with LLMs doesn’t hold, because the grammar of asking LLMs to write code is not only non-deterministic but fundamentally unexplainable. It’s a neutral net with millions upon millions of weights. The designers of LLMs can’t even explain how a particular output comes about; they were swearing blind that they couldn’t regurgitate full pictures and books until a bunch of security researchers proved them wrong.
Nice article. This really matched my experience.
I have almost zero formal coding background, but I’ve used AI to hack together a few small apps purely as a learning exercise. I’m not trying to ship anything or sell it, which probably changes the equation quite a bit.
Reading both the article and the comments, it’s clear how much background matters. I can see why experienced engineers find AI coding frustrating. Brittle output, strange mistakes, and the last 10% still being very real work. If you care about clean, maintainable code, that would get old fast.
But for me, coming at this with a "beginner’s mind," though, it’s been a lot of fun. I expect things to break. When they do, I adjust the prompt, learn a little more, and keep going. No pressure, no deadlines, no illusion that the AI actually “knows” anything.
For learning and exploration, it’s been a surprisingly enjoyable ride. Bugs, hallucinations, and all.
I guess some people are always on the cross."This opinion comes not as an endorsement but as personal experience"
Yeah, but this isn't your personal blog, though. This is just like those youtubers that go "This is not advertisement because nobody's paying me!" Ah so it's advertising you do for free, cool, cool.
Also: Not ONE mention of ethical concerns. Not ONE. Very, very interesting.
This is the problem at the root of so many industries. Bosses that got to be boss by talent playing the boss game, not by talent doing the actual thing the company does. They are constantly looking for New Initiatives, because that's how that class keeps score. You get enough initiatives completed and you level up. Doesn't matter if those initiatives were really needed or are successful in the long term. In fact, it's fine if they're not, because the next boss or two that comes along will need to start an initiative to rip out the old thing the previous bosses added and replace it with their own great new initiative. It's a cycle that just repeats in so many businesses.That reminds me of my boss ~10 years ago who wanted to know how we could add HTML5 to our product. Our 100% locally-installed WinForms/WIN32 product.
While the first 90% is easy in many ways, in others it's actually quite hard. It's laying the foundations for that last 10%, and if the foundations are built poorly that last 10% is going to be much worse. It's an easy job for those that have the expertise to do it. I think AI has not been proven to have that expertise, just as many junior coders have not. That's why even though it's the easy part, it's important to have experience and skill involved at that stage.I found the opposite. I use it to do the easy 90% of the work, leaving the challenging 10% for me to finish.
But I do the same at work too. I only take the bugs and features that no one else is willing to touch. Or I assist when somebody else gets stuck. So I'm used to somebody else doing the easy part.
Literally? Everyone?? Hm.Literally everyone saying what you are all say the same thing:
"I'm a hobbyist programmer and it's great for me, so that means it's going to be great for everyone"
Very narrow-minded approach.
Also, why the heck do you AIbros all talk the same? Do you also use the LLM to construct your replies?
I think Python is a good analogy. The language came along and allowed all kinds of people without a software engineering background to write code that solved a problem for them, significantly lowering the barrier to entry.
More superficial doublespeak. The whole thing can be summed up in one quote, and kudos to PW for calling it out:Just came across this survey from PricewaterhouseCoopers
https://www.pwc.com/gx/en/issues/c-suite-insights/ceo-survey.html
I think Python is a good analogy.
Ah yes, the conspiracy theory of (checks notes) the Ars Technica AI reporter being told he should write some stories about using AI.I guess some people are always on the cross.
Have you never heard of an opinion piece in news media? That said, I strongly suspect Benj is being directed by Conde Nast(or, well, someone) to write these articles, and they do really seem to be more of an advertisement more often than not. However, in an opinion piece, I'm fine accepting large amounts of bias.
In a standard news article, I am not OK with that same level of bias. And this is not a standard news article. So to me, it is fine, as I know what I'm getting into by reading it.
And why would ethics (a valid concern) even come into play here, considering this is an opinion piece?
Personally, my "conspiracy theory" is that Conde Nast often forgets Ars even exists. This is in no way to diminish Ars' excellent work (especially since it's my #1 go to website for the things it covers), but they seem like one of the smaller fishes in CN's pond. They have like a half billion customers worldwide. Ars seems like a few drops in the bucket on that scale, and one that is not recognizable in mainstream culture.The features are pitched internally, they're edited, it's a normal thing all the writers do. Writing about personal experiences is a great way to round out hitting daily news and providing depth and interest you can't get from another outlet.
The only thing that's weird here is that people are even trying to make a thing out of the most basic daily operations.
Those people will hopefully work on niches and nooks that were previously deemed too labor intensive for custom software. Ideally we are moving to more fit for purpose (small scale) software, since (arguably) programming just got cheaper.I really hope you're correct, but my thought process has been that if it allows people to be more productive, there will typically be layoffs. If a company has 1000 developers and they find the tool allows each of them to even be 15% more productive, they can easily justify 100+ people being let go. They might find more work for them, but in my experience when they can save money, they will.
Out of all the conde nast-related conspiracy theories I have seen posted to the Ars comment section since the acquisition in 2008, this one is closest to reality, lol. Generally the only time anyone at Ars who isn't a senior manager communicates with anyone at conde is when they need to file an expense report or during benefits open enrollment in November.Personally, my "conspiracy theory" is that Conde Nast often forgets Ars even exists.
Really glad you enjoyed playing it! Of all the game ideas I cooked up, I find myself playing that one the most.I spent some fun time with my youngest kid playing Violent Checkers. This also reminds me I need to polish my Tron game with super abilities. I guess it's a good opportunity to give another chance to these coding agents.
Yes indeed! Believe me, nobody told me to get sick with COVID for a month and go nutty making non-stop game demos using AI tools for 6-8 hours a day during my winter vacation. That was entirely my own madness.Ah yes, the conspiracy theory of (checks notes) the Ars Technica AI reporter being told he should write some stories about using AI.
How do people think this all works? Like the daily flow of news you read? Is it a grand conspiracy every time a writer is assigned a story to you? Or when a writer here states an opinion on something?
This is the gig. Benj was hired to write about AI, that's what he does. Sometimes he has his own story ideas based on the daily flow of news, sometimes an editor says "hey you should hit the news about Apple going with Gemini" or whatever, and sometimes he plans out a more long form feature like "what if I did a piece on trying AI code for a month?"
The features are pitched internally, they're edited, it's a normal thing all the writers do. Writing about personal experiences is a great way to round out hitting daily news and providing depth and interest you can't get from another outlet.
The only thing that's weird here is that people are even trying to make a thing out of the most basic daily operations.
Thank you for saying this. It has never made sense to me either that an LLM trained on GPLv3 code and which can be tricked into producing GPLv3 output fairly simply can produce code which is anything other than GPLv3 licensed. It is very much a derivative work.However, I don't think it's unreasonable to argue that models trained on GPLv3 code must be GPLv3 themselves.
You're not wrong.1. grammar asking for code to be written is nondeterministic
2 grammar to convey intent is fundamentally unexplainable
3. intent being conveyed to a neural network (non-artificial) with millions upon (some very large number of) millions of weights.
4. large numbers of neuroscience research careers into (biological) neural networks can't explain how a particular output comes about
-and a bonus-
5. asking the (biological) neural network why it produces an output generates, at best, a post-hoc, self serving narrative created out of whole cloth.
yep! sounds exactly like the process of getting squishy humans to write code
I hope you told them "Sure thing!" and then slapped the HTML5 logo in a corner of an obscure splash-screen somewhere. I mean, technically that would be adding HTML5 to your product!That reminds me of my boss ~10 years ago who wanted to know how we could add HTML5 to our product. Our 100% locally-installed WinForms/WIN32 product.