Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about it
https://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek Compare how cryptographic RNGs are usually Pseudo-RNGs fed with entropy, and which fail to output random-approximate values (of a given strength) once the entropy falls too low.
It's almost as if there is a pattern to this.
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
i'm here for the inevitable model collapse. let's immanentize this bitch!
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek So... let me get this straight. Autocoprophagic #RSI •doesn't• lead to #AGI? Say it ain't so!
#AI -
@devsimsek Is that a thing people believe, that LLMs generate themselves towards the singularity simply by eating their own output and no other feedback?
@Quantensalat @devsimsek I'm sure you'll find plenty of straw men who do
-
@TallSimon @devsimsek I haven’t looked at the proof, but I wonder if Gödel plays a role in it. Seems like at least Gödel would strongly imply this new proof.
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek this is one of those things that seemed intuitive to us skeptics but it's great to see it proven
-
@Quantensalat @devsimsek I'm sure you'll find plenty of straw men who do
@dpiponi @devsimsek I find the paper interesting but I would like to understand the exact
premises. "AI" is not equal to gen AI or LLMs, it probably makes little sense to sell it as a general statement about "AI" -
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek wow, almost as if this was a problem known as overtraining for well over 30 years
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek it's the only thing that makes sense if you know just a little about how they work (I don't know more than a little)
Like if you output whatever is most likely, and input that again, it's only logical (at least to me) that eventually you'll get a mushy average -
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek This feels like a weird argument, because it proves a version that I've never heard anyone arguing for. Like, when I've heard people talk about AI itself accelerating AI's improvement (on both pro and con sides), the argument wasn't that AI would self-train on its own output. The argument was that AI would replace AI developers and accelerate the development of better AI code.
-
@devsimsek wow, almost as if this was a problem known as overtraining for well over 30 years
@SRAZKVT Exactly.
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek Nobody ever claimed that llms get better by being trained on their own synthetic data. This blog post is very misleading.
The idea of self-improvement and singularity is that llms write improved versions of their own codebase and perform the research and experiments for coming up with better models themselves.
The idea of singularity is interesting but also full of hidden assumptions. I'm always confused when people act like singularity would exist. It's just science fiction. -
@dpiponi @devsimsek I find the paper interesting but I would like to understand the exact
premises. "AI" is not equal to gen AI or LLMs, it probably makes little sense to sell it as a general statement about "AI"@Quantensalat @dpiponi That's what I hate about these companies.
-
@devsimsek So... let me get this straight. Autocoprophagic #RSI •doesn't• lead to #AGI? Say it ain't so!
#AI@ghostinthenet Yep
-
@devsimsek also see https://berryvilleiml.com/2026/01/10/recursive-pollution-and-model-collapse-are-not-the-same/
This is part of a long running #ML research thread with big #MLsec impact
@noplasticshower Thanks, ill look into it
-
@devsimsek Not a developer, but that was my first thought when I understood how LLMs were trained and how they worked: What happens when there's so much AI generated content on the internet that the LLM is harvesting and recycling its own output? That's like a high school history class having their own essays as research material. #LLMs
@anne_twain YEP, most of the people whom commented assumes they develop every iteration with fresh data; that comes from internet ...
-
@Quantensalat @dpiponi That's what I hate about these companies.
@devsimsek @dpiponi that they act like AI=LLMs?
-
@devsimsek Not a developer, but that was my first thought when I understood how LLMs were trained and how they worked: What happens when there's so much AI generated content on the internet that the LLM is harvesting and recycling its own output? That's like a high school history class having their own essays as research material. #LLMs
@anne_twain @devsimsek
"That's like a high school history class having their own essays as research material." - a memorable phrase. -
@devsimsek @dpiponi that they act like AI=LLMs?
@Quantensalat @dpiponi yes. I did used the same tactic while naming my post as satire. its annoying....
-
@devsimsek @dpiponi that they act like AI=LLMs?
@Quantensalat @devsimsek There's a setup around equations (9) and (10) where the distribution used for training the next generation is a linear combination of the distribution your current generation generates and external data. As the amount of external data goes to zero, you expect model collapse. This is hardly surprising. I don't know anyone who expects you can just keep training based on previous results and expect something radically new to happen. (Though something *useful* can happen - eg. you may improve performance this way. See "rectification" in flow-matching.)
Note that this doesn't rule out all forms of self-training - just one kind. As a concrete example, an LLM trained to generate code can learn from the output of the generated code. Such output is, in some sense, exogenous.
