Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
-
@devsimsek Hi,
I just wanted to ask what the difference between this and fitting a simple regression model on predicted outcomes? I feel the conclusions of this study is pretty obvious even from a simple regression case - or is it not? Why would we expect something different from LLMs?@drmambobob hi
Before I reply completely you must know that I'm not an expert on this topic but just a curious guy.You made a great remark, it is essentially the same problem scaled up. The obvi part is that you cannot get something from nothing. The main reason I assume the paper authors are talking about it differently with llms is that while a regression model just gets stale, olm goes probabilistic hell. It basically actively deletes its output diversity until it produces a single acceptable response.
I hope this answers your question.
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about it
https://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek I am the farthest thing from a mathematician but just from a general lay persons perspective the thesis seems a little rigid. Assuming all training data outside the internet is corrupt, assuming that synthetic data only comes from recursive souring, etc. I don't really want to cheerlead AGI as I assume it will bring about a bunch of existential risk but I'm still in the "wouldn't bet against it" camp.
-
-
@devsimsek I am the farthest thing from a mathematician but just from a general lay persons perspective the thesis seems a little rigid. Assuming all training data outside the internet is corrupt, assuming that synthetic data only comes from recursive souring, etc. I don't really want to cheerlead AGI as I assume it will bring about a bunch of existential risk but I'm still in the "wouldn't bet against it" camp.
@mike Yeah I see your pov. I just made some remarks and some of them were satirical which you can obviously see through my writing style. I do agree with your stance on rigidity.
Also thanks for the comment
I hope you have enjoyed scrolling through those gifs -
@mike Yeah I see your pov. I just made some remarks and some of them were satirical which you can obviously see through my writing style. I do agree with your stance on rigidity.
Also thanks for the comment
I hope you have enjoyed scrolling through those gifs @devsimsek Forgive me as I digress. Even though I left Twitter years ago whenever I even mildly challenge someone on social media I still "brace for impact" after I press send. It's so refreshing to get a polite and coherent response that resembles civil discourse. Mastodon still amazes me.
-
@mike Yeah I see your pov. I just made some remarks and some of them were satirical which you can obviously see through my writing style. I do agree with your stance on rigidity.
Also thanks for the comment
I hope you have enjoyed scrolling through those gifs @devsimsek @mike I have a vague feeling that there's an argument to be made that a model that can train itself which is then given continuous IO from/to the world (vision, hearing, touch, voice, body) might be able to evolve into a self-sustaining intelligent entity, as long as it has to work for its survival (i.e. work for its input energy).
-
@devsimsek @mike I have a vague feeling that there's an argument to be made that a model that can train itself which is then given continuous IO from/to the world (vision, hearing, touch, voice, body) might be able to evolve into a self-sustaining intelligent entity, as long as it has to work for its survival (i.e. work for its input energy).
@virtuous_sloth @mike On a personal belief I don't think so but this must be tried or at least done an experiment.
I believe someone out there are currently working on it
-
@virtuous_sloth @mike On a personal belief I don't think so but this must be tried or at least done an experiment.
I believe someone out there are currently working on it
@devsimsek @mike I certainly don't think we are anywhere near it nor will we be until someone invents a silicon nerve (simple multiple-input binary output tunable arbitrary functional nerves) that uses very little energy instead of simulated (via linear algebra on GPUs) ones.
-
@devsimsek Forgive me as I digress. Even though I left Twitter years ago whenever I even mildly challenge someone on social media I still "brace for impact" after I press send. It's so refreshing to get a polite and coherent response that resembles civil discourse. Mastodon still amazes me.
@mike always, I am open for feedback that's how I learn at least
Also similar thing happened on reddit; someone called my blog post ai generated, I told them nope its all written by me and they apologized. That's not normal
At least not according to my experience -
@devsimsek @mike I have a vague feeling that there's an argument to be made that a model that can train itself which is then given continuous IO from/to the world (vision, hearing, touch, voice, body) might be able to evolve into a self-sustaining intelligent entity, as long as it has to work for its survival (i.e. work for its input energy).
@virtuous_sloth @mike well that sounds intriguing
-
@devsimsek @mike I certainly don't think we are anywhere near it nor will we be until someone invents a silicon nerve (simple multiple-input binary output tunable arbitrary functional nerves) that uses very little energy instead of simulated (via linear algebra on GPUs) ones.
@virtuous_sloth @devsimsek There is a technology that is using light pathways to create an "Analog" GPU which is showing some promise.
-
@virtuous_sloth @devsimsek There is a technology that is using light pathways to create an "Analog" GPU which is showing some promise.
@mike @devsimsek Interesting.
Also confusing since GPUs are currently being used to do massively-parallel floating-point arithmetic as part of the matrix-matrix and matrix-vector math involved in the linear algebra of the models, and there's not much analog about that I am aware of. I do know that simple resister, capacity, & inductor circuits are analog representations of 2nd-order differential equations, but not sure how that would apply here.
/shrug/ -
@mike @devsimsek Interesting.
Also confusing since GPUs are currently being used to do massively-parallel floating-point arithmetic as part of the matrix-matrix and matrix-vector math involved in the linear algebra of the models, and there's not much analog about that I am aware of. I do know that simple resister, capacity, & inductor circuits are analog representations of 2nd-order differential equations, but not sure how that would apply here.
/shrug/@virtuous_sloth @devsimsek This is the company, however there's not much on their web site. I believe I saw a youtube video explaining its operation. The gist was that the major roadblock was analog to digital conversion to utilize memory. Apologies my memory of the details is vague. https://qant.com/photonic-computing/
-
@virtuous_sloth @devsimsek This is the company, however there's not much on their web site. I believe I saw a youtube video explaining its operation. The gist was that the major roadblock was analog to digital conversion to utilize memory. Apologies my memory of the details is vague. https://qant.com/photonic-computing/
@mike @devsimsek
OK, so it looks like regular linear algebra but using photonics (one of my PhD committee members pioneered in the field of photonics https://en.wikipedia.org/wiki/Sajeev_John) for much higher energy efficiency, presumably by trading digital exactness (hah! IEEE 754 is not exact) for analog computation.Their current produce is 8 GOPS (but does that mean GFLOPS?) and current GPUs are doing TFLOPS, but they seem confident about 1000-fold improvements every 1.5 years, so 2027 here they come!
-
@mike @devsimsek
OK, so it looks like regular linear algebra but using photonics (one of my PhD committee members pioneered in the field of photonics https://en.wikipedia.org/wiki/Sajeev_John) for much higher energy efficiency, presumably by trading digital exactness (hah! IEEE 754 is not exact) for analog computation.Their current produce is 8 GOPS (but does that mean GFLOPS?) and current GPUs are doing TFLOPS, but they seem confident about 1000-fold improvements every 1.5 years, so 2027 here they come!
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
Love it when the mathematical model puts in black and white what most of us intuitively pointed out. Just proves our billionaire geniuses are short on genius and long on shell games.
-
@anne_twain @devsimsek
This reminds me of some right-wing Youtube channels or Telegram groups. For years they're feasting on their own output and in the process are getting intellectually shallower but more agitated and radical... -
@Quantensalat @devsimsek the main issue is that unless you maintain an external signal (so human input in the form of token sequences that are actually carefully curated for coherence) the models become more and more incoherent. Sounds like you're on board with that. The next step is that we're quickly devaluing money spent on human creativity and the world is awash in LLM garbage. So the human signal *is* disappearing.
@wronglang @Quantensalat @devsimsek On the reverse side, we have humans generating human output and then asking LLM chat bots to "improve" that output. What the chat bot deems wrong or improvable causes the elimination of creative personal human voice. Our unique voice gets contaminated and reformed to conformity.
.