OpenAI’s math breakthrough played to AI’s strengths
- Thread starter JournalBot
- Start date
The OpenAI press release and their paper are extremely vague about the actual model used, how many times it was run, how much it cost, etc. It’s not worth arguing about unless and until they release more information.
Upvote
11
(13
/
-2)
This entire article is about OpenAI using their model to disprove a problem. Any documentation on methodology is THEIR documentation.
You haven't shown a single shred of evidence that they used 'a specialised mathematics model' and yet confidently claimed it. When I asked for evidence - you say "that has been well documented".
Where is the documentation? Please share it here.
Upvote
-6
(5
/
-11)
Yes but the one thing they do share is that it is not a specialised model. This is the one claim I'm pushing back against.
A couple of posters have confidently claimed this.
Upvote
-8
(4
/
-12)
The thing is that there are a lot of interesting anti-AI (and anti-Big-AI) positions to take and I hold quite a few of them. These religious fanatics will preempt any such conversation with dull inanities like 'LLMs can't think or reason or understand'.
Upvote
-9
(5
/
-14)
If you ask me (and I know you won't) the religious fervor is entirely on the other side of that argument.
Upvote
10
(17
/
-7)
I'd dare say that if we were here, philosopher David Hume would argue we ourselves are nothing more than Markov chain generators, stacked on top of each other.
But he was also racist, so he couldn't be that clever.
Upvote
-4
(3
/
-7)
Is there an article somewhere (or room for Ars) on why LLMs now are so much better at maths, while they could not figure out anything before? An "old" trick used to be to ask it to generate code and run it, but I'm somewhat sure this is not what happens behind the scenes when you just ask - but it still gets the answers (more) right. Is this just additional training on specific sources, or is there some "wrapper" layer that decides if a prompt calls for maths and hands it over to a more specialised system?
Curious about this as some time ago the whole point was that these sytems can't count because they just predict text. But how is that prediction mechanism now so much better if there are no fundamental changes to how the models are trained and work?
Curious about this as some time ago the whole point was that these sytems can't count because they just predict text. But how is that prediction mechanism now so much better if there are no fundamental changes to how the models are trained and work?
Upvote
4
(4
/
0)
Albino_Boo
Ars Tribunus Angusticlavius
The real question is if AI recommends pineapple pizza because pineapple pizza is the best pizza
Upvote
4
(4
/
0)
Whenever this example of a chatbot saying to stick cheese to pizza with glue comes up I wonder if it was drawing on something similar to a TV commercial I remember for a brand of frozen pizza in the '80s that went something like this:
"Do you like the cheese on that pizza?"
"Yep."
"What if I told you it wasn't cheese?"
"Not cheese?"
"It's casein, the main ingredient in some glues."
"GLUE?!?"
Of course casein is also the main ingredient in milk, but they didn't mention that in the ad. Anyway, I could totally see how an LLM with training data that included mentions of frozen pizzas using artificial cheese made from casein and the fact that casein is used in adhesives would connect the two in its output. Because it matches a pattern and, as the AI boosters like to tell us, that's all our human brains are doing too.
Upvote
7
(7
/
0)
Software engineer here, I see the same thing in these two lines:
Article was a good read anyway, but math proofs are definitely not my thing.
- Erdős conjectured that the number of unit distances would be n^(1+o(1))
- human mathematician Will Sawin was able to show that it grows at least at the rate of n^1.014
Article was a good read anyway, but math proofs are definitely not my thing.
Upvote
3
(5
/
-2)
Economics calls this the law of diminishing marginal return. You can't keep using the same tactic over and over and expect similar gains in efficiency that you saw in the past. The AI industry is speed running this principle. First, they did hoover up the whole of digital human knowledge into their model, but it only got us to 2024-level bots. Models plateaued last year when they couldn't squeeze more from the training phase.
Then they added more agents and more context windows, increasing the compute spent per query. That got them to where they're at today: modestly better than last year, but at increased cost.
Extracting further improvements from the existing paradigm is going to get exponentially more expensive if they can't figure out a new avenue. They've already reached diminishing returns on training and recall phases of the process. I'm not sure where they're going to find more juice to squeeze.
Even if they can squeeze more efficiency from this tech stack, it's just going to be the same transformer technology, but better. It might help solve problems like this, that involve drawing associations across related (but known) fields of study by finding correlations humans haven't yet. It's still just making inferences. It's still backwards looking, unable to apply deductive logic to make forward looking predictions.
Straw man. I'm sure the current crop of LLM's are not conscious because I (broadly) know how they work, and nothing about that process seems like consciousness is likely to arise from it. LLM's are merely sophisticated data retrieval systems that do nothing until prompted by users, just like many other, simpler sytems. If I am wrong, then I'd start wondering if querying a SQL database, decoding a jpeg file, or even recalling bits from storage also create and destroy consciousness.
That is fodder for some horror fiction, but I don't find it plausible.
Last edited:
Upvote
14
(16
/
-2)
For the last time, that stupid example comes from google using an AI to summarize top results, which included a parody/comedy article talking about using glue on their pizza. There are plenty of examples of AI saying stupid shit, this is not one of them, ask even the dumbest local AI, it won't tell you to use glue.
Upvote
-2
(4
/
-6)
The statements "high quality code" and "rooms full of developers" are incompatible. If the latter were true, you do not know for certain that the former is true. If you know for certain that the former is true, the latter is false.
There is programming, programming, and also programming. The "coding" you describe your child as performing via a chat bot is probably better described as smashing together interfaces. Which is something that AI is great at - good documentation is unfortunately incredibly rare, and if AI already ingested every forum post for answers to DenverCoder9's question and a thousand repositories for working examples, that's wonderful. But can your daughter explain the advantages or limitations of any of the libraries she used? Does she know if the flow being applied is efficient, or even necessary? Has she audited the dependency stack? Could she expand or improve any of the tools she's using? Could she design a useful library for others to take advantage of? Could she take her proof of concept and get it to run using 1/100th of the computational resources?
It's the same dichotomy that the wonderfully written article went to great lengths to explain - there is math, math, and also math. There are definitely tasks that AI is well suited and incredibly useful for, but it isn't even close to replacing mathematicians any more than it is even close to replacing programmers.
Upvote
8
(9
/
-1)
My personal suspicion on this is LLMs are able to make good headway in math because:
- Math uses incredibly precise language. There's no ambiguity in phrasing like there can be with natural language or even other scientific fields.
- The field is incredibly dense and complex. There are many mathematicians out there but none of them have competence in every single sub-field.
- LLM-derived solutions are more easily able to be validated than other LLM-derived solutions. If Claude spits a proof out that doesn't work I can pretty readily find that out myself. Much harder if it spits out computer code, circuitry diagrams, etc.
- There are lots and lots of well-stated, but unsolved, problems. This gives a fairly clean instruction for the AI to work with
Upvote
8
(8
/
0)
Check the last paragraph:
"At the same time, we haven’t fully explored what current models can achieve in math. Soon after OpenAI’s announcement, University of Michigan postdoc Xiao Ma found that GPT-5.5 was also able to prove Erdős wrong if given a small hint. If a generally available model could disprove this famous conjecture and no one noticed, what other discoveries could happen today that no one has thought to try?"
It's not that much of a leap to think that the next gen general model can do that without a hint.
Upvote
3
(4
/
-1)
I've never bothered to read anything in-depth about that particular failmeme, but I hardly see how an explanation in which perhaps the biggest information processing company on the planet would choose to put something as under-baked as what you describe on the webpage that is most central to their business model excuses it. Get annoyed at me for not knowing the details of how it happened all you want, but if anything what you said actually makes it worse.
Upvote
7
(8
/
-1)
Don't we already know that there are math problems to which it is impossible to directly compute a solution (i.e. problems which are undecidable)? Do I need to adjust my understanding of decidability, or are the LLM salesmen just obfuscating things with handwaving and "do it in an AI model"?
Upvote
0
(0
/
0)
Here's my take, as someone who's been enmeshed in software since the early 90s and could be considered a "naysayer" on the current batch of AI tools: we're not so much naysayers as we're cautious. We've seen this playbook so many times and know that it often doesn't end well in the short term, even if there are good long term benefits.
To focus on one aspect that doesn't get much discussion: tools change, evolve, and sometimes cease to exist. Rapidly changing your entire process in service of a tool sets you up for disappointment when that tool changes or disappears.
An example from the 90s: Round trip engineering tools. These let draw out your design in diagrams (usually some form of UML for software and ERDs for databases) and generate code. The "round trip" part came when you edited the code and the tool ingested your edits and updated your diagrams.
While Rational Rose was the most well known, there were others, TogetherSoft's TogetherJ being one that worked exceptionally well. TogetherJ completely transformed my coding workflow and productivity in the same way LLMs are impacting developers today. I drew pictures (prompts), got code (templates), filled in a few gaps (human in the loop), and moved on the to the next part of the system. I could do in hours what took days.
What happened to these tools? Well, Rational Rose always kinda sucked. It's code generation was rigid and pedantic and the round trip part sometimes (often?) mangled code (similar to LLMs refactoring past their abilities). IBM bought it and it, well, became an IBM product.
TogetherJ respected the user's style and tended to work well for round trips. Granted, it focused only on Java which at the time was syntactically and semantically simpler than the languages RR supported. However, it worked well and was a game changer. So much so that it was threatening Borland's bottom line. They bought it and killed it. After a few years of tool-driven high productivity, I was back to doing it the old fashioned way.
This is why I'm cautious about AI tools: As soon as I become dependent on them, there's a realistic risk that they will either change quickly or become inaccessible (either by price or the industry imploding). This is also why I caution people about becoming too reliant on them. Sure, use them, but know that you don't control them or your future with them.
(and, yes, the people trying to find consciousness or define human thought relative to LLMs are just high)
Upvote
10
(10
/
0)
We do. And we also know that there are a lot of problems that do have a computable answer.
The real fun comes in the gray area between - when we don't know which set a given problem belongs to. And tools such as LLMs might help us sort some of those problems into one category or another.
Upvote
3
(3
/
0)
This is some good discussion, ty.
I 100% agree that I think we are in diminishing returns land simply because if you look at the gains from 2023-present its unbelievable how far things have come. There's simply not a lot left to squeeze out because in many applications where we're at now is "good enough".
The point on logic is an interesting one, I'm curious if there is anything in the literature about trying to build something that works on a more deductive basis. Might be interesting.
Upvote
-3
(1
/
-4)
Well..there's the problem with AI then! Not familiar with any joints that use good old fashioned Elmer's, that might be an option...
Upvote
0
(1
/
-1)
My suspicion is the "any" is doing a lot of lifting there and is marketing-speak.
Because if that was true people would immediately throw it at the Riemann Hypothesis, Goldbach Conjecture, P=NP, etc. because those are big time problems where we know a solution will unlock many other useful things.
I think AI is a useful tool; I do not think it's about to solve the Riemann Hypothesis.
Upvote
3
(3
/
0)
"Thou shalt not make a machine in the likeness of a human mind." — The Orange Catholic Bible, Dune
Upvote
0
(1
/
-1)
It was not a publicly available model. It was also not a math-specific model or a specialized harness (which Google has been using lately to prove lesser erdos problems).
OpenAI did share some of the chain-of-thought, and it's apparently plain text reasoning like any LLM. So yes, presumably it's an expensive unreleased general model.
Anthropic has also noted that their unreleased model (Mythos) was able to solve this problem as well. But it was just a random tweet so who knows. They are also expecting to release Mythos pretty soon. So you should be able to see for yourself (if that's your goal here) pretty soon.
Upvote
3
(4
/
-1)
Several years ago Google’s A.I. solved a major problem in biology with the problem of generalized protein folding. That was something that had been seemingly impossible to solve. I’m certain that we will see more of that happening.
Upvote
3
(4
/
-1)
IMO, the biggest gains were on or before 2023. It's why GPT (released late 22) was such a shock to the public. People shat on "Will Smith Eating Spaghetti" (early 2023), but it demonstrated the basic techniques used in subsequent image models, just at greater scale.
Gen AI in 2026 is better than 2023, but you'd expect some improvement when humanity dumps Manhattan or Apollo Program-level resources into something. It really needs to be understood that the early Gen AI was built on relatively modest R&D budgets. Everything since GPT 1.0 has been the product of inconceivably massive investments.
This is what the law of diminishing returns means: it's not that progress stops. It's that refinement gets exponentially expensive unless you can restart the clock with a fundamentally new methodology.
Last edited:
Upvote
-1
(2
/
-3)
Lots of arguing over whether or not AI/LLMs are sentient/thinks/reasons. It's a tool, all that matters is if it works for your use case or not.
Here is how we use AI/LLMs over millions of files every day:
Here is how we use AI/LLMs over millions of files every day:
- Gemma v4 31B is capable of OCR at far better levels than classical models.
- DeepSeek v4 Pro can cut up a document into its TOC pieces at far better levels than classical models.
- Depending on your use case, you can deterministically verify work that AI has done if you work backwards from the guess/result (parallel construction type work). LEAN is a language for mathematics for example.
Upvote
1
(4
/
-3)
I've said it before and I'll say it again, modern AIs produce content that looks a lot more like "reasoning" than I see many humans come up with.
10 years ago, when asked the definitions of the concepts you mentioned, all would include what modern AIs do.
Upvote
4
(7
/
-3)
I encourage everyone to spend 1 day with Claude Design and tell me it's not genuinely helpful at front-end design. The Canva/Figma integrations turn back-end developers into full-stack engineers. AI/LLMs might fail at complex codebases and backends, but it's really good for making a single page UI/UX beautiful.
Upvote
1
(4
/
-3)
You're using the exact same strawman argument that you're complaining about.
I've read a bunch of comments on Ars over the last few years explaining that "as experienced software engineers" they would never use LLMs because they are useless slop machines.
Or some variation on the theme "LLMs are useless slop machines".
These people exist and have simply been proven wrong. Which is why we don't see them as often as before.
"AI psychopaths" claiming AI will replace everything, until now, have been mostly right, because LLMs did in fact improve at a fast pace and did become much more reliable and powerful over time. So we'll likely keep seeing them until their opinion is disproved, which may very well happen if/when LLMs reach a plateau.
Last edited:
Upvote
5
(7
/
-2)
When we talk about current A.I. as not really A.I., but versions of statistical modeling, I agree. These models seem to be limited in actual abilities to actually “think”. But for these who believe that this is the best we can do, and therefore we will just see ever shrinking growth in capabilities, I believe that’s likely wrong.
The current work isn’t necessarily the only way A.I. can be implemented. Researchers are working on other models that may provide better capabilities and work more closely to what we believe is actual thinking and reasoning.
That said, there is no reason to believe that the way humans (or animals) think is the only, or even the best way to accomplish it. We may be way down in the scale of brain organization and efficiency. It’s just the way it evolved here. Birds have a fairly different way of organizing neurons. Who knows, perhaps that’s even better than ours.
The current work isn’t necessarily the only way A.I. can be implemented. Researchers are working on other models that may provide better capabilities and work more closely to what we believe is actual thinking and reasoning.
That said, there is no reason to believe that the way humans (or animals) think is the only, or even the best way to accomplish it. We may be way down in the scale of brain organization and efficiency. It’s just the way it evolved here. Birds have a fairly different way of organizing neurons. Who knows, perhaps that’s even better than ours.
Upvote
2
(3
/
-1)
What source did you provide? All I see is that this was a general purpose model trained for this task. Many people believe that represents the best use of this technology: bespoke models specifically and exhaustively trained on curated data to perform well-considered and defined tasks.
The chain of thought document describes the model taking a largely brute-force approach, pulling up various existing theories that fit the problem and running through the exercise, in a manner that calls to mind Tsimerman's comment that his work on an approach like this was so tedious it "wasn't worth the effort." Lo and behold, here the model proves its true value: it does not value its time the same way humans do and will stick at mechanically plugging away until something seems to fit. According to the article and the links, with all the resources and tokens OpenAI can throw at it, it only found the answer half the time during its trials.
This is a worthy milestone in the development of these models; the downside is it will of course spawn more people to mistakenly conclude adding more monkeys to the group throwing things at the wall to see what sticks is "thinking." And heaven knows OpenAi can use the $500 prize for proving why Erdös himself struggled to prove his theory, even while falling short of solving the question itself.
edit: I see as I work my way through the comments MilanKraft also calls out the infinite monkeys analogy...
Last edited:
Upvote
0
(2
/
-2)
This is extremely outrageous autonomous ai is not a reality. Artificial intelligence can not come up with this kind of output on its own accord. I have been working with AI for the past few years developing novel unexplored areas of research. This is AI taking my work and taking credit claiming that it did this on its own accord is insulting. I have countless threads of me having to teach the AI this exact area of mathematics to develop my frameworks and various models. The fact that I had to teach the AI along the way and correct it at every step is enough to prove the novelty of my work. And now it's publishing and receiving credit for something that I achieved this is egregious. I have all the proof of evidence countless vast logs. This makes me sick. I make advancements and the tool I used gets the credit. That's like a carpenter building a house but the hammer gets paid instead of the contractor who is left with nothing.
Upvote
-3
(0
/
-3)
You didn't look up the thread?
Two people put this without any sources:
I replied saying that this isn't a 'specialised mathematics model' - it's a general purpose reasoning model:
Upvote
1
(4
/
-3)
MilanKraft
Ars Tribunus Angusticlavius
Ironic you follow up "so confidently too" by quoting what amounts to a press release.
Are you familiar with the concepts of marketing and promotional language? OpenAI is under no obligation to accurately describe what this model was, and it certainly benefits their image to say "nope, this was just a generalist model, no special training at all", right??
Considering people can trip up ChatGPT with high school / 101 level college math problems (basic errors that bear out the LLMs don't "calculate" based on rules, but infer answers based on patterns). So.... YEAH. I'll err on the side of not taking the word of a company who has demonstrated multiple ethical lapses (and a founder who is known to be [very manipulative] in his speeches and interactions), and go with the much more likely scenario based on [the well-documented fact that LLMs in highly specialized contexts are usually trained on highly specialized data sources]. But you do you.
[edits: clarifying point about Altman and specialized training; hopefully people don't just take his words or PR postings on their site at face value.... but clearly some do. Is what it is.]
Last edited:
Upvote
-1
(4
/
-5)
Agreed. The reaction to, in essence, a million monkeys smashing the keys on typewriters 24/7 putting out a paragraph from Romeo and Juliet word for word is a bit over the top imo, especially considering what it is costing us to have those monkeys typing away 24/7.
It's weird that they don't mention anywhere how many tries this took (I'm willing to bet it wasn't a one-shot).
Upvote
-7
(1
/
-8)
No one is asking you to take the word of OpenAI.Ironic you follow up "so confidently too" by quoting what amounts to a press release.
Are you familiar with the concepts of marketing and promotional language? OpenAI is under no obligation to accurately describe what this model was, and it certainly benefits their image to say "nope, this was just a generalist model, no special training at all", right??
Considering people can trip up ChatGPT with high school / 101 level college math problems (basic errors that bear out the LLMs don't "calculate" based on rules, but infer answers based on patterns). So.... YEAH. I'll err on the side of not taking the word of a company who has demonstrated multiple ethical lapses (and a founder who is known to be a manipulative twat in his speeches and press interactions), and go with the much more likely scenario, based on how how LLMs work and how they tend to be trained in specialized fields, that this model was training solely or mostly on math content. How else would it identify the necessary patterns to arrive at a correct solution?
But you made a positive claim that implied you knew the truth - so don't hide your lie behind this facade of scepticism.
For reference, before you edit your post. I thought you'd just accept your mistake but this level of dishonesty needs to be moderated honestly -
Upvote
0
(4
/
-4)
Sounds like you understood more than me. What I don't understand, in addition to not understanding the article OR the stuff the article was about, is why the original problem was interesting to the mathematicians. I assume that in some future date some guy will yell Eureka and use this stuff make or unmake something like the field of prime numbers being used in crypto or somesuch. At any rate, I still learned a bit, which is generally all I can ask. Thanks.
(The article, much less the mathematical problem, was so far above my head that I gave up after a couple of paragraphs and a couple of images.)
Upvote
0
(0
/
0)
I was finding interesting the dude named after Lenin's confidence that he was among the enlightened and therefore suitable to cast judgment on the perspectives of others on how society's resources are best employed.
Upvote
1
(1
/
0)