Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
-
@musicman @devsimsek As with all mathematical theorems, there's probably a not too far-fetched loophole circumventing some of their assumptions, doesn't mean skynet is becoming self-aware any time soon once that is the case.
@Quantensalat @musicman @devsimsek depends on what you mean by far fetched, certainly nothing as easy as "their more compute at it' which is what made this jump in investment so dramatic.
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about it
https://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek so it doesn't get stuck in a local optimum, it hill-climbs a non-existent one?
-
@anne_twain @devsimsek
"That's like a high school history class having their own essays as research material." - a memorable phrase.@knowattitude Thank you.
-
@Quantensalat @devsimsek tech bros have been claiming their AIs are alive for years so if the average person who knows nothing about computers thinks we already have AGI, who can really blame them. Anthropic all but claims to have invented Terminator.
Maybe something like this will stop the panic.
Which is not to say people shouldn't be concerned in general and very specifically about environmental impacts
@musicman @Quantensalat @devsimsek Anyone who ever copied an audio tape (or worse a VHS tape) knows that the copy is always worse than the original. And in the video case, soon unwatchable.
Ever heard a repeating echo on a video meeting that just turns to a buzz? Same phenomenon.
So what you need is an AI that can perform experiments in the real world to learn how to do better whatever it is you want it to do.
Inbreeding animals doesn't work too well either.
-
@anne_twain @devsimsek this requires two components LLMs do not, cannot, and will not ever have. Intent and originality.
Researchers have done self-modifying targeted things. It takes no time at all for things to become impossible for humans to understand. This does not mean they are better. Usually they weren't. Even when hyper-focused with specific controls.@rootwyrm @devsimsek At present, as I understand it, humans are required for several steps of the process - programming, data input (training), prompting and most importantly, evaluating the output and acting/not acting on it.
The last step doesn't get talked about very much but is crucial. The large number of anecdotes of "AI fails" we see every day show that *humans are deciding* whether the output is relevant, likely to be correct, appropriate etc. and whether and how it should be acted on. So the human is acting as a "crap filter" and ... what shall we call it ... activator? Agent? The people selling LLMs don't want us to see this, they want us to see a magic box -
@rootwyrm @devsimsek At present, as I understand it, humans are required for several steps of the process - programming, data input (training), prompting and most importantly, evaluating the output and acting/not acting on it.
The last step doesn't get talked about very much but is crucial. The large number of anecdotes of "AI fails" we see every day show that *humans are deciding* whether the output is relevant, likely to be correct, appropriate etc. and whether and how it should be acted on. So the human is acting as a "crap filter" and ... what shall we call it ... activator? Agent? The people selling LLMs don't want us to see this, they want us to see a magic box@anne_twain @devsimsek there is no process. There is no intelligence. There never was and there never will be.
It's a bad stochastic parrot written by children who should have been flunked out of 7th grade math and 3rd grade English as illiterate. Used and pushed by people who aren't capable of reviewing a fast food order, or even placing one.And guess what? All irrelevant because it takes an incomprehensible level of stupidity to even use a tool that fails dangerously constantly.
-
@anne_twain @devsimsek there is no process. There is no intelligence. There never was and there never will be.
It's a bad stochastic parrot written by children who should have been flunked out of 7th grade math and 3rd grade English as illiterate. Used and pushed by people who aren't capable of reviewing a fast food order, or even placing one.And guess what? All irrelevant because it takes an incomprehensible level of stupidity to even use a tool that fails dangerously constantly.
@anne_twain @devsimsek a better equivalence explanation.
Here is a 'smart hammer.' It promises to never smash your thumb. And between 20 and 60% of the time, it works! The other 80 to 40% of the time it explodes and takes off your entire arm and sets the nearest three houses on fire.
The question is not "why are people not stopping when it explodes" or "how do we filter the explosions."
The question is "WHY THE FUCK ARE PEOPLE STILL USING AN EXPLODING HAMMER?!"I need to remember this one.
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek I'd be interested to see the same analysis of human consciousness. It is well understood that complexity is a regime on the absolute edge of chaos.
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek This & overall the bigger issue of forced overinclusion & attempted hyperteliance on machine learning systems, mostly done by governments & their private partners, like autoshutoff on cars, chatbots as talk therapists& biometric ID/digital ID instead of regular ID card systems, is destined to fail.... It's not so much that activists will win in court or public protests on how these things at least mostly violate civil liberties & are based on data & intellectual property theft.... It's that fundamentally none of these systems actually work!
They couldn't even write a specific mechanism or method for the vehicle one because nothing fitting the mandate has been developed & the nearest ones obviously dont work.
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek you have an awkward sentence here you might want to know about: “Even though I like to say yes, i neither have the enough research nor I want to comment on it”
I think you’re going for something like “even though I’d like to say yes, I have neither enough research nor any desire to comment on it”… but I’m not entirely sure.
-
@Quantensalat @musicman @devsimsek depends on what you mean by far fetched, certainly nothing as easy as "their more compute at it' which is what made this jump in investment so dramatic.
@wronglang @musicman @devsimsek No, agreed, more compute with the same type of model and the same training data sounds totally unplausible to me as a long term strategy
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek
"Touch grass." It is not just a reminder to take a break or get some fresh air. -
Large language models are fundamentally different from mammals on every level. They do not build models or reason about them. A rat is more "intelligent".
-
@Quantensalat @devsimsek the main issue is that unless you maintain an external signal (so human input in the form of token sequences that are actually carefully curated for coherence) the models become more and more incoherent. Sounds like you're on board with that. The next step is that we're quickly devaluing money spent on human creativity and the world is awash in LLM garbage. So the human signal *is* disappearing.
@wronglang @devsimsek Yes, sure. I mean I can imagine it improving somewhat still, like when you augment your training set for image recognition by adding noise to a smaller set, but only to a point before it goes downhill from feedback.
No, my gut feeling is rather that there have to be much more effective ways to train a model than to brute force funnel billions of pages of text to a transformer which blindly fits relations between words and structures without understanding them, that seems like doing it the hard way, even if I'm not expert enough to tell you what an alternative would look like
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
So. That kind of "AI" - if not refused by us - shall vampirize humanity forever. It is build to suck our data forever. It is the perfect tool for control freaks. By design.
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek It's logical really as AI simply doesn't have any intelligence. It only regurgitates what it has stolen and learnt.
-
@devsimsek Not a developer, but that was my first thought when I understood how LLMs were trained and how they worked: What happens when there's so much AI generated content on the internet that the LLM is harvesting and recycling its own output? That's like a high school history class having their own essays as research material. #LLMs
@anne_twain @devsimsek
This reminds me of some right-wing Youtube channels or Telegram groups. For years they're feasting on their own output and in the process are getting intellectually shallower but more agitated and radical... -
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek the problem is: absolutely nothing you and I, all the world's scientists or anyone left with a sane mind says will stop the AI hype train because there's too much money in it already.
-
@devsimsek Nobody ever claimed that llms get better by being trained on their own synthetic data. This blog post is very misleading.
The idea of self-improvement and singularity is that llms write improved versions of their own codebase and perform the research and experiments for coming up with better models themselves.
The idea of singularity is interesting but also full of hidden assumptions. I'm always confused when people act like singularity would exist. It's just science fiction. -
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
Interesting article about how AI cannot grow into a super intelligence because the more systems grow, the more they rely on information generated by themselves and the more ....
'it forgets what reality looks like'.
#AI #AGI #RSI
(RSI = Recursive Self-Improvement)
https://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
