Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
-
@anne_twain @devsimsek
This reminds me of some right-wing Youtube channels or Telegram groups. For years they're feasting on their own output and in the process are getting intellectually shallower but more agitated and radical... -
@Quantensalat @devsimsek the main issue is that unless you maintain an external signal (so human input in the form of token sequences that are actually carefully curated for coherence) the models become more and more incoherent. Sounds like you're on board with that. The next step is that we're quickly devaluing money spent on human creativity and the world is awash in LLM garbage. So the human signal *is* disappearing.
@wronglang @Quantensalat @devsimsek On the reverse side, we have humans generating human output and then asking LLM chat bots to "improve" that output. What the chat bot deems wrong or improvable causes the elimination of creative personal human voice. Our unique voice gets contaminated and reformed to conformity.
-
@Quantensalat @devsimsek the main issue is that unless you maintain an external signal (so human input in the form of token sequences that are actually carefully curated for coherence) the models become more and more incoherent. Sounds like you're on board with that. The next step is that we're quickly devaluing money spent on human creativity and the world is awash in LLM garbage. So the human signal *is* disappearing.
@wronglang @Quantensalat @devsimsek On the reverse side, we have humans generating human output and then asking LLM chat bots to "improve" that output. What the chat bot deems wrong or improvable causes the elimination of creative personal human voice. Our unique voice gets contaminated and reformed to conformity.
-
@wronglang @Quantensalat @devsimsek On the reverse side, we have humans generating human output and then asking LLM chat bots to "improve" that output. What the chat bot deems wrong or improvable causes the elimination of creative personal human voice. Our unique voice gets contaminated and reformed to conformity.
@jeawhiz @Quantensalat @devsimsek that's a great point--I agree completely! Here's a list of reasons why...
-
@mike @devsimsek
That being said, I have no idea how real or how actually useful it is (but since apparently it can be bought and they have Python libraries, it is not vaporware).@virtuous_sloth @devsimsek Another European company working on photonic computing. One of the partners at the end says something to the effect that if they're successful there will be no need for cloud AI as you could simply run models locally. He should be careful with those kinds of statements when most of the US economy right now is a bet on cloud AI. However I'm becoming more convinced that if they can get it right that this is the future. https://www.youtube.com/watch?v=9tqOPS6x9l8&t=3s
-
@devsimsek Is that a thing people believe, that LLMs generate themselves towards the singularity simply by eating their own output and no other feedback?
@Quantensalat @devsimsek@universe
Well not with that attitude they won't. Why, they just simply need to pull themselves up by their own bootstraps!
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about it
https://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
Is this related to Strange Attractors?
-
@Quantensalat @devsimsek@universe
Well not with that attitude they won't. Why, they just simply need to pull themselves up by their own bootstraps!
@Enema_Cowboy Attention is all they need
-
@virtuous_sloth @devsimsek Another European company working on photonic computing. One of the partners at the end says something to the effect that if they're successful there will be no need for cloud AI as you could simply run models locally. He should be careful with those kinds of statements when most of the US economy right now is a bet on cloud AI. However I'm becoming more convinced that if they can get it right that this is the future. https://www.youtube.com/watch?v=9tqOPS6x9l8&t=3s
@mike @devsimsek Heh, I'm confident most of the US investment community will do their best to ignore causes of and potential warning signs of collapse, regardless.
If anything, if people were to take his words seriously then it would only help because the longer things go on, the stronger the collapse, I think.
-
@virtuous_sloth @devsimsek Another European company working on photonic computing. One of the partners at the end says something to the effect that if they're successful there will be no need for cloud AI as you could simply run models locally. He should be careful with those kinds of statements when most of the US economy right now is a bet on cloud AI. However I'm becoming more convinced that if they can get it right that this is the future. https://www.youtube.com/watch?v=9tqOPS6x9l8&t=3s
@mike @virtuous_sloth that's so intriguing. I have to check that out.
Thanks
-
@mike @virtuous_sloth that's so intriguing. I have to check that out.
Thanks
-
Is this related to Strange Attractors?
@Enema_Cowboy weird question, I haven't thought of them while I was reading the article as well but as far as I remember strange attractors was on rnns and rnns only. I might be mistaken though.
Also thanks for the comment, hope you liked reading the post
-
@Enema_Cowboy weird question, I haven't thought of them while I was reading the article as well but as far as I remember strange attractors was on rnns and rnns only. I might be mistaken though.
Also thanks for the comment, hope you liked reading the post
I was thinking about strange attractors because of Edward Lorenz and Margaret Hamilton's contributions to the study of chaos theory. Lorenz was modeling atmospheric convection and discovered that the model would degrade because small inconsistencies would magnify over successive iterations. The resulting data formed a strange attractor (Lorenz Attractor).
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek The Baron Von Munchausen effect?
https://mythcrafts.com/2025/03/28/an-impossible-idiom-pulling-yourself-up-by-the-bootstraps/ -
@dpiponi @devsimsek I find the paper interesting but I would like to understand the exact
premises. "AI" is not equal to gen AI or LLMs, it probably makes little sense to sell it as a general statement about "AI"@Quantensalat
I've collected some articles on AI nomenclature you might find useful:https://tech.lgbt/@toolbear/116446645444544147
The one by Ali Alkhatib I found to be the most illuminating.
@dpiponi
@devsimsek -
I was thinking about strange attractors because of Edward Lorenz and Margaret Hamilton's contributions to the study of chaos theory. Lorenz was modeling atmospheric convection and discovered that the model would degrade because small inconsistencies would magnify over successive iterations. The resulting data formed a strange attractor (Lorenz Attractor).
That's intriguing, thanks for showing your chain of thought
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
Central to this hypothesis is the concept of recursive self-improvement: an AI system with the capacity to inspect and enhance its own architecture or training processes would initiate a positive feedback loop, with each generation of the AI being more intelligent than the last, leading to exponential growth in its capabilities.
TL;DR: as always, the headlines are over-simplified to the point of inaccuracy...
They're proving the second half, that model collapse due to self-training is inevitable.
But they're not really addressing the first half, leaving open the question of whether an AI system could improve its own code or architecture. I personally don't think an LLM designing a better LLM gets us any closer to the so-called singularity, either, but that's just an opinion.
Of course, the industry (at least in the US) has gone all in (and then mortgaged the house and pawned its jewelry and sold its blood and firstborn and bet that, too) on just making the model bigger and the feeding it more data. So it's useful to have proof that the current strategy deadends when they run out of real human data.
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek and meanwhile them companies are buying all the processors and memory they can find pretending that’s the way to AGI. Whatever that is, btw.
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek nice, how long does it take?
-
M monkee@chaos.social shared this topic
M monkee@other.li shared this topic
