Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
-
I'm sorry, did I hurt your feels by pointing out that you're choosing emotions over facts?
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about it
https://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek Hi,
I just wanted to ask what the difference between this and fitting a simple regression model on predicted outcomes? I feel the conclusions of this study is pretty obvious even from a simple regression case - or is it not? Why would we expect something different from LLMs? -
I'm not going to respond to abuse with my own abuse, no matter how tempted I am.
This is "abuse":
"anything else is hallucinated by the researchers fooled by [...]"
Don't debate subjects you don't understand. Your initial post to me is laughably ignorant and directly contradicted by science:
"[LLMs] do not build models or reason about them"
-
This is "abuse":
"anything else is hallucinated by the researchers fooled by [...]"
Don't debate subjects you don't understand. Your initial post to me is laughably ignorant and directly contradicted by science:
"[LLMs] do not build models or reason about them"
-
What? I didn't even mention or refer to you in the text you quoted.
-
@devsimsek Hi,
I just wanted to ask what the difference between this and fitting a simple regression model on predicted outcomes? I feel the conclusions of this study is pretty obvious even from a simple regression case - or is it not? Why would we expect something different from LLMs?@drmambobob hi
Before I reply completely you must know that I'm not an expert on this topic but just a curious guy.You made a great remark, it is essentially the same problem scaled up. The obvi part is that you cannot get something from nothing. The main reason I assume the paper authors are talking about it differently with llms is that while a regression model just gets stale, olm goes probabilistic hell. It basically actively deletes its output diversity until it produces a single acceptable response.
I hope this answers your question.
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek I am the farthest thing from a mathematician but just from a general lay persons perspective the thesis seems a little rigid. Assuming all training data outside the internet is corrupt, assuming that synthetic data only comes from recursive souring, etc. I don't really want to cheerlead AGI as I assume it will bring about a bunch of existential risk but I'm still in the "wouldn't bet against it" camp.
-
-
@devsimsek I am the farthest thing from a mathematician but just from a general lay persons perspective the thesis seems a little rigid. Assuming all training data outside the internet is corrupt, assuming that synthetic data only comes from recursive souring, etc. I don't really want to cheerlead AGI as I assume it will bring about a bunch of existential risk but I'm still in the "wouldn't bet against it" camp.
@mike Yeah I see your pov. I just made some remarks and some of them were satirical which you can obviously see through my writing style. I do agree with your stance on rigidity.
Also thanks for the comment
I hope you have enjoyed scrolling through those gifs -
@mike Yeah I see your pov. I just made some remarks and some of them were satirical which you can obviously see through my writing style. I do agree with your stance on rigidity.
Also thanks for the comment
I hope you have enjoyed scrolling through those gifs @devsimsek Forgive me as I digress. Even though I left Twitter years ago whenever I even mildly challenge someone on social media I still "brace for impact" after I press send. It's so refreshing to get a polite and coherent response that resembles civil discourse. Mastodon still amazes me.
-
@mike Yeah I see your pov. I just made some remarks and some of them were satirical which you can obviously see through my writing style. I do agree with your stance on rigidity.
Also thanks for the comment
I hope you have enjoyed scrolling through those gifs @devsimsek @mike I have a vague feeling that there's an argument to be made that a model that can train itself which is then given continuous IO from/to the world (vision, hearing, touch, voice, body) might be able to evolve into a self-sustaining intelligent entity, as long as it has to work for its survival (i.e. work for its input energy).
-
@devsimsek @mike I have a vague feeling that there's an argument to be made that a model that can train itself which is then given continuous IO from/to the world (vision, hearing, touch, voice, body) might be able to evolve into a self-sustaining intelligent entity, as long as it has to work for its survival (i.e. work for its input energy).
@virtuous_sloth @mike On a personal belief I don't think so but this must be tried or at least done an experiment.
I believe someone out there are currently working on it
-
@virtuous_sloth @mike On a personal belief I don't think so but this must be tried or at least done an experiment.
I believe someone out there are currently working on it
@devsimsek @mike I certainly don't think we are anywhere near it nor will we be until someone invents a silicon nerve (simple multiple-input binary output tunable arbitrary functional nerves) that uses very little energy instead of simulated (via linear algebra on GPUs) ones.
-
@devsimsek Forgive me as I digress. Even though I left Twitter years ago whenever I even mildly challenge someone on social media I still "brace for impact" after I press send. It's so refreshing to get a polite and coherent response that resembles civil discourse. Mastodon still amazes me.
@mike always, I am open for feedback that's how I learn at least
Also similar thing happened on reddit; someone called my blog post ai generated, I told them nope its all written by me and they apologized. That's not normal
At least not according to my experience -
@devsimsek @mike I have a vague feeling that there's an argument to be made that a model that can train itself which is then given continuous IO from/to the world (vision, hearing, touch, voice, body) might be able to evolve into a self-sustaining intelligent entity, as long as it has to work for its survival (i.e. work for its input energy).
@virtuous_sloth @mike well that sounds intriguing
-
@devsimsek @mike I certainly don't think we are anywhere near it nor will we be until someone invents a silicon nerve (simple multiple-input binary output tunable arbitrary functional nerves) that uses very little energy instead of simulated (via linear algebra on GPUs) ones.
@virtuous_sloth @devsimsek There is a technology that is using light pathways to create an "Analog" GPU which is showing some promise.
-
@virtuous_sloth @devsimsek There is a technology that is using light pathways to create an "Analog" GPU which is showing some promise.
@mike @devsimsek Interesting.
Also confusing since GPUs are currently being used to do massively-parallel floating-point arithmetic as part of the matrix-matrix and matrix-vector math involved in the linear algebra of the models, and there's not much analog about that I am aware of. I do know that simple resister, capacity, & inductor circuits are analog representations of 2nd-order differential equations, but not sure how that would apply here.
/shrug/ -
@mike @devsimsek Interesting.
Also confusing since GPUs are currently being used to do massively-parallel floating-point arithmetic as part of the matrix-matrix and matrix-vector math involved in the linear algebra of the models, and there's not much analog about that I am aware of. I do know that simple resister, capacity, & inductor circuits are analog representations of 2nd-order differential equations, but not sure how that would apply here.
/shrug/@virtuous_sloth @devsimsek This is the company, however there's not much on their web site. I believe I saw a youtube video explaining its operation. The gist was that the major roadblock was analog to digital conversion to utilize memory. Apologies my memory of the details is vague. https://qant.com/photonic-computing/
.