AI costs how much? GitHub Copilot users react to new usage-based pricing system

Some report burning through their whole monthly "AI credit" allotment in a single day.

See full article...

 

[AI_Skeptic]

Everyone on earth is pricing in 10x YoY cost decreases, and even using a model 6 months behind frontier (minimax) yields even greater savings than that, so it’s truly insane to see people not aggressively leaning into the curve.

Give me a source of how training costs decreases 10x YoY that's NOT from a private company.

 

[AI_Skeptic]

By no one sharing them online anymore?

Once a model, or a software program, is open source, and people start using the software and creating forks, I never seen the software program "fade away". At worst it just goes on archive.org

 

[benabena]

At least Evernote is still free.

 

[smallscreensw]

Prices are going to go up everywhere - and keep going up - so users that think "going somewhere else" is an option are in for a surprise.

Doesn't mean the technology is going away. But people aren't going to use these things as readily as they do now once the massive subsidies go away.

But they will be able to go "somewhere else", just not necessarily a blue-chip or premium service. Whether that ultimately winds up suiting their purposes will vary from user to user, and from budget to budget. I looked at the Hugging Face site yesterday and there are a LOT of choices for LLMs out there.

 

[WereCatf]

Once a model, or a software program, is open source, and people start using the software and creating forks, I never seen the software program "fade away". At worst it just goes on archive.org

I do have to point out that you not being aware of it doesn't mean it has never happened.

 

[shadedmagus]

Have you ever been to Hugging Face? You can find the original OSS-20b model at https://huggingface.co/openai/gpt-oss-20b . Because it's free and open source, people take the model, and create their own versions of it.

I have never seen an active open source project that was pulled off line like that. Can you provide a link to one?

If I'm understanding @Madestjohn correctly, he's worried that the free model clearing houses will be pushed offline by the big houses in order to force people to subscribe to their UBB hell products.

But I've been around a while, too - and while e.g. Nintendo has succeeded in driving most of the free game ROM sites out, you can still find their ROMs for download.

I wouldn't be surprised if the free models don't migrate in a similar way. And if that's the case, I definitely won't be surprised if "guerilla" model training becomes a thing.

 

[JohnDeL]

That's nothing really new though. Ed Zitron calls them business idiots (though I think he got that term from somewhere else). The business world is filled with them. Especially at larger companies where size and momentum can paper over a lot of incompetence.

Yeah.

The problem with upper management is that it tends to be a generational wealth thing; you get to be CEO because your father was a CEO (even if it was in a different company) or because you were in the same fraternity as a board member or...

Even if the inheritance tax were 100%, this would still persist and still be a problem.

 

[2023SpaceXOddessy]

The technology used for SMS/ text messaging is completely differently, and prices can be reduced. I can not see how pricing can be reduced for complex LLM usage. Can you explain how cost can be lowered using actual examples with sources?

The fact it took so long for phone providers to switch to an unlimited amount of texts, despite the fact it never actually cost, as much as they charged their customers for a single text SMS was never originally designed for text messaging. There were actual costs to sending text messages on the older networks.

With AI it seems we had the reverse happen, the real costs were hidden to us, and were first given the "unlimited' option only to discover that paying per text message is the future.

 

[archtop]

The issue still stands. If all the meat bags are unemployed then there are no customers to buy whatever widgets companies produce. Worse still the level of benefits governments now need to pay all the unemployed can't be covered by PAYE and the like so company tax will need to rise to support all the unemployed. Companies will be paying a huge tax bill to support meatbags they do not employ and will have no customers as the meatbags are too poor to buy their stuff.

Looks a somewhat sub optimal situation tbh

Plumbers and electricians will support the economy.

 

[disarmyouwitha]

Hey, until OpenAI runs our of money — just start using Codex xD Tibo does an API limit reset like every other day. xD

 

[apousada]

Anyone managing enterprise accounts can simply setup a budget for "All AI Credit SKUs" to prevent getting a huge, surprise bill.

See: docs.github.com/en/enterprise-cloud@latest/billing/how-tos/set-up-budgets

 

[dio82]

The million (billion? trillion???) dollar question is how much does it actually cost to run these models. I think companies like Anthropic and OpenAI earn a big profit on customers who pay per token. I'm looking forward to their IPOs so that I can see how their finances really look like.

No. They are running on insane, eye watering, jumping off skyscrapers level of loss. And best thing? The costs scale. The more users the higher the costs per user.

 

[spamthejabroni]

Everyone seems to be focusing on raw scaling and yeah, with current transformer architectures maybe we have hit a wall in terms of data and horsepower. But scaling is not the only way technologies improve and I don’t believe we’re already at the end of improvement, that what’s been achieved in the last ~5 years is the end of AI development.

But I’m not going to continue discussion only for my posts to be hidden simply due to being unpopular so whatevs, believe what you will.

(please downvote this into oblivion too as I only want those of you who would do so to read it)

"please downvote this into oblivion"

Uh, ok.

 

[Geebs]

But they will be able to go "somewhere else", just not necessarily a blue-chip or premium service. Whether that ultimately winds up suiting their purposes will vary from user to user, and from budget to budget. I looked at the Hugging Face site yesterday and there are a LOT of choices for LLMs out there.

Hugging Face is VC-backed to the tune of several hundred million dollars as well as an undisclosed amount from Amazon and Meta. It’s revenue is in the tens of millions of dollars i.e. it’s incredibly unprofitable. Model weights run from tens to hundreds of gigabytes per download. Serving that amount of data is expensive. Some guy with a patreon isn’t going to be able to host these models if the big boys decide they don’t want to share any more.

 

[AI_Skeptic]

I wouldn't be surprised if the free models don't migrate in a similar way. And if that's the case, I definitely won't be surprised if "guerilla" model training becomes a thing.

Have you used a Chatbot Frontend like Jan or GPT4All before (or went directly to Hugging Face)? If you haven't, check it out, and then go to the models section. People take the open source models, and since they are open source, modify them.

I'm sure we all used ChatGPT, and asked it a question that borderlines naughty, and ChatGPT says "sorry, I can't do that". People take those models, and modifies them so it can't refuse to give an answer (i.e. the OpenAi-GPT-Oss-20b-HERETIC-Uncensored-NEO-Imatrix-Gguf model at https://huggingface.co/DavidAU/OpenAi-GPT-oss-20b-HERETIC-uncensored-NEO-Imatrix-gguf)

So if Hugging Face goes out of business, GitHub, Archive.org will still host those models. And I don't think it's possible for an open source project to suddenly change the license to close source without making modifications to the code.

 

[AI_Skeptic]

Some guy with a patreon isn’t going to be able to host these models if the big boys decide they don’t want to share any more.

What about GitHub? What about Archive.org?

 

[breze]

The fact it took so long for phone providers to switch to an unlimited amount of texts, despite the fact it never actually cost, as much as they charged their customers for a single text SMS was never originally designed for text messaging. There were actual costs to sending text messages on the older networks.

With AI it seems we had the reverse happen, the real costs were hidden to us, and were first given the "unlimited' option only to discover that paying per text message is the future.

The transition from mostly-expensive to mostly-unlimited texting was a while ago, though. This “reverse” thing is just the modern startup model. Everything is sold at a loss initially to get network effects and hype. It’s really weird, it seems like an huge chunk of the tech sector orbits around this “anti-competitive pricing as a service.”

LLMs are a maybe the apotheosis of this, but every streaming service, every messaging app, every social media network, it’s all built with the intent of the enshitification cash-in.

Telecos are have one foot in infrastructure, one in digital… so, who knows how that’d shake out nowadays. But mostly the had the misfortune of starting up in a era where companies had to sell things profitable to prove that they had a market…

 

[AI_Skeptic]

I do have to point out that you not being aware of it doesn't mean it has never happened.

I'm here to learn. Can you please provide an example of an open source program, that's been forked, and used by at least 100K people, that was fully shut down and can't be downloaded from another site?

 

[klarg]

The true cost is even higher when you factor in the higher electric power costs borne by the general public (due to higher demand from data centers, and compounded by the lower rates utilities give data centers (and other large users) which have to be offset somehow (by higher costs to residential consumers as the go-to move). And by tax deals (abatements, rebates, ad nauseam) for data centers which shift an appreciable burden to other property owners. And future costs, such as the cost associated with abandoned hulks of data centers which will inevitably dot the landscape just as empty malls do now.

 

[s73v3r]

I really don't think this is a sign of a bubble popping. (Well, not for the AI providers, but maybe some businesses that depended on cheap tokens.)

So, the AI providers. Because without cheap tokens, there aren't customers.

It shows that there is actually a ton of demand for AI workloads

There's a ton of demand at a highly subsidized price. There's no evidence that demand will continue at the actual price.

, enough that they can afford to fire their worst customers.

You mean the ones actually using it.

The ones who are left will be getting actual value for their money.

Will they? Because the only way this works for the AI companies is that the people left, who are paying, don't use it.

I thought they were going to have to make like those bitcoin miners in Texas who make money by not mining. These massive datacenters could hold the power grid, water supplies, and the climate hostage for cash.

And that's why they shouldn't be allowed to be built.

 

[arsisloam]

Everyone seems to be focusing on raw scaling and yeah, with current transformer architectures maybe we have hit a wall in terms of data and horsepower. But scaling is not the only way technologies improve and I don’t believe we’re already at the end of improvement, that what’s been achieved in the last ~5 years is the end of AI development.

But I’m not going to continue discussion only for my posts to be hidden simply due to being unpopular so whatevs, believe what you will.

(please downvote this into oblivion too as I only want those of you who would do so to read it)

Complaining about down votes is the surest way to farm down votes.

You know you're a smart person. You don't need external validation from random people on the internet. Who cares what other people think.

 

[AI_Skeptic]

I wouldn't be surprised if the free models don't migrate in a similar way. And if that's the case, I definitely won't be surprised if "guerilla" model training becomes a thing.

One of the main differences between OpenAI et.al and Nintendo is that Nintendo never made a software program that was under an open source license. For example, Super Mario Brothers is close source, and the code and copyright is completely owned by Nintendo. OpenAI's GPT-OSS-20b model is under Apache 2.0 license, allowing ANYONE to share, modify, distribute, and sell O) the software.

So OpenAI can pull GPT-OSS-20b (and other models) off from Hugging Face, but if Github or Archive.org or a random website wants to host it, they have no legal standing to stop it.

 

[gosand]

This reminds me of the pricing scheme of SMS/text messaging in the nascent cell-phone days.

I worked at Motorola in the early 90s. I still remember my phone plan - $0.11 a minute. That was it. There was no SMS in the early days.

This does remind me of it a little bit, except then people needed to get the phones. It took a while for enough people to have them to make SMS and later features desirable.

Anyone who is surprised by this has never heard of the company who owns github/copilot.
Will other companies follow suit? Only if they like money.

 

[s73v3r]

It's just the way fundamental technologies go. I would rather ask what could stop it.

No. Stop fucking handwaving this away, and be specific about how it's going to get better. 3DTV was a hot technology, and tell me how that's gotten better in the past few years?

There is literally nothing inherent about technology that means it will always get better.

I have no crystal ball but more data

Where are they going to get that from?

, more and faster training and inference hardware and techniques

Where are those going to be put?

new types of models and architectures

Like what, exactly?

There are millions of people using it and thinking about how to make it better. It will get better.

Again, there is literally nothing inherent about this stuff that says it has to keep getting better.

In tech time LLMs are brand new, we're just scratching the surface.

They are not. They have been around for quite some time. Stop using that excuse.

 

[Prostetnic]

It's odd that my AI costs have not changed at all. They are still stubbornly stuck at $0.00

 

[AI_Skeptic]

a two-year old LLM is no practical use to anyone.

What purpose are people using LLMs?

 

[s73v3r]

Right at the start of the AI coding “revolution” there was a sliver of a chance that LLMs could be taught to write optimised machine code and unlock the wasted potential of everyone’s existing hardware, with all of the attendant gains in UX and energy saving.

Instead, we have a bunch of ludicrously expensive server racks churning out tens of thousands of lines per day of plagiarised javascript.

That chance never existed, based on the way these things work.

 

[pug fugly]

What? You thought you were going to be the benefactor of AI productivity gains? Silly pleb.

 

[s73v3r]

Everyone seems to be focusing on raw scaling and yeah, with current transformer architectures maybe we have hit a wall in terms of data and horsepower. But scaling is not the only way technologies improve and I don’t believe we’re already at the end of improvement, that what’s been achieved in the last ~5 years is the end of AI development.

But I’m not going to continue discussion only for my posts to be hidden simply due to being unpopular so whatevs, believe what you will.

(please downvote this into oblivion too as I only want those of you who would do so to read it)

Again with the handwaving. You have an almost religious faith that someone will rescue you. It is just as likely that there is no way this technology gets to a state where it's cost effective.

Your posts are being downvoted to oblivion because you keep making claims with literally no evidence to back them up.

 

[AI_Skeptic]

But I’m not going to continue discussion only for my posts to be hidden simply due to being unpopular so whatevs, believe what you will.

I'm here to learn. I provide links to why I think how I think. If what you're saying is true, those links should be easy to find.

 

[zaghahzag]

God forbid people realize the cost of the thing they've been using. This was, of course, inevitable.

All that VC / hyperscaler money that had been getting burnt on customer acquisition and experimenting was going to demand ROI at some point.. and that point is now.

2026: The year the AI bill came due.

It turns out you can't run a massive cash burn hopes-and-dreams machine forever without consequences. It's no small coincidence that that OpenAI / Anthropic are rushing to IPO before the costs catch up with everyone waking up to the real costs of the LLMs.

I actually wonder if this is Microsoft's way of throwing a wrench into the IPOs or trying to get a cheaper price for itself.

does anyone know how the copilot subscriptions are working behind the scenes? They're mostly using openai or anthropic models behind the scenes. Was microsoft setting it's own sub prices and then turning around and paying API rates?

 

[Aswansong]

There are a number of journalists and commentators who have been trying to discuss how AI is unaffordable, and will never be sustainable in its current form (costs outweigh profits by several multiples) and outlets like this have been ignoring them and relentlessly pumping the AI hype train. It’s pathetic.

When the whole market crashes, I hope some of you will look in the mirror and understand that your role is meant to be holding truth to power - not running their marketing arms.

 

[AI_Skeptic]

and outlets like this have been ignoring them and relentlessly pumping the AI hype train

How does this website pump the AI hype train?

 

[Control Group]

One of the main differences between OpenAI et.al and Nintendo is that Nintendo never made a software program that was under an open source license. For example, Super Mario Brothers is close source, and the code and copyright is completely owned by Nintendo. OpenAI's GPT-OSS-20b model is under Apache 2.0 license, allowing ANYONE to share, modify, distribute, and sell O) the software.

So OpenAI can pull GPT-OSS-20b (and other models) off from Hugging Face, but if Github or Archive.org or a random website wants to host it, they have no legal standing to stop it.

This.

If the big companies want to vanish the already released and open-licensed models and then succeed, they should pivot to selling their magic to Sony, Steam, EA, etc. They might actually turn a profit that way.

Doom: Eternal is bigger than any of the models I run. And it’s illegal to distribute on top of that. And still, I have the sneaking suspicion I could find a cracked copy if I really wanted to.

 

[TekaroBB]

How does this website pump the AI hype train?

There are have been a bunch of positive or uncritical AI articles and at least a couple of the authors here are a least somewhat AI enthusiasts. But they've also hosted a ton of AI critical articles, so I'd say they are at most neutral? There certainly is not an AI overlord in the management team demanding more AI positive articles to "relentlessly" hype it up.

 

[jeffbax]

that kind of subsidized customer acquisition may soon give way to Copilot-style usage-based pricing across the industry

Sorry but this framing is disingenuous. Before there were no agents doing actual work, and now there are.

That’s the rub.

The previous plans were not built for what these things are capable of doing now (aka building apps end to end)

 

[jlpatte2]

One person can do the work of three people, for the cost of four! By Grabthar's Hammer, what a savings.

Best reply of the whole bunch! Well played sir! Well Played! You win post of the day!

 

[AI_Skeptic]

Doom: Eternal is bigger than any of the models I run. And it’s illegal to distribute on top of that. And still, I have the sneaking suspicion I could find a cracked copy if I really wanted to.

Exactly! Let's say the worst thing happens and the AI companies pull the models from Hugging Face, and send threatening letters to GitHub, Archive.org, et.al. There are peer-to-peer websites where questionable files can be downloaded, hosted on peer-to-peer. Unlike Disney and other companies, OpenAI, et.al can not sue a person downloading those files because they already released it under the Apache 2.0 license.

Since Apache 2.0 license allows anyone to re-sell the software as well, I'm sure there will be websites where you can order a USB stick with the files already on the drive as well, if the bandwidth is too expensive.

 

[AI_Skeptic]

Before there were no agents doing actual work, and now there are.

I never seen agents that were able to do work. Please provide a source of agents that can do work, correctly, and a more efficient method can't be used.