Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
-
Wouldn't it be prudent if you learnt anything about the subject first?
Here - I'll help: One of the better books on the subject is "Consciousness: An Introduction" by Susan Blackmore.
I read it 15 years ago. That you believe there's a "platform" and "software" means you have no idea how human cognition works.
I have been following this approach since the first steps in the '80s. I'm pretty clear on how it works.
Software is a metaphor, the connectome is obviously a different kind of construct than procedural code, but it is the connections, not the fact that it is built out of neurons, that determines the kind of reasoning and model construction that the human brain performs. You are looking at the implementation and ignoring the big picture.
-
I have been following this approach since the first steps in the '80s. I'm pretty clear on how it works.
Software is a metaphor, the connectome is obviously a different kind of construct than procedural code, but it is the connections, not the fact that it is built out of neurons, that determines the kind of reasoning and model construction that the human brain performs. You are looking at the implementation and ignoring the big picture.
Here - read a scientific paper:
"Our findings reveal reasoning-like mechanisms within the LLM's layers that operate across structurally similar tasks. Crucially, these mechanisms remain stable despite variations in input and output data, suggesting the existence of internal processes that transcend basic language processing."
There's no hardware and software in humans. The hardware and the software are one and the same.
https://www.sciencedirect.com/science/article/pii/S2949882126000010
-
Here - read a scientific paper:
"Our findings reveal reasoning-like mechanisms within the LLM's layers that operate across structurally similar tasks. Crucially, these mechanisms remain stable despite variations in input and output data, suggesting the existence of internal processes that transcend basic language processing."
There's no hardware and software in humans. The hardware and the software are one and the same.
https://www.sciencedirect.com/science/article/pii/S2949882126000010
All the "internal processes" are linguistic, and not even sophisticated linguistic processing, anything else is hallucinated by the researchers fooled by the "clever hans" effect.
All mammals have basically the same hardware. All behavioral differences are due to differences in the size and arrangement of the connections between the neurons.
-
All the "internal processes" are linguistic, and not even sophisticated linguistic processing, anything else is hallucinated by the researchers fooled by the "clever hans" effect.
All mammals have basically the same hardware. All behavioral differences are due to differences in the size and arrangement of the connections between the neurons.
-
Your argument is "when my gut feeling doesn't agree with science I trust my gut feeling"?
I mean. That's a choice.
-
Ah, abuse. It always comes down to abuse.
-
I'm sorry, did I hurt your feels by pointing out that you're choosing emotions over facts?
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about it
https://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek Hi,
I just wanted to ask what the difference between this and fitting a simple regression model on predicted outcomes? I feel the conclusions of this study is pretty obvious even from a simple regression case - or is it not? Why would we expect something different from LLMs? -
I'm not going to respond to abuse with my own abuse, no matter how tempted I am.
This is "abuse":
"anything else is hallucinated by the researchers fooled by [...]"
Don't debate subjects you don't understand. Your initial post to me is laughably ignorant and directly contradicted by science:
"[LLMs] do not build models or reason about them"
-
This is "abuse":
"anything else is hallucinated by the researchers fooled by [...]"
Don't debate subjects you don't understand. Your initial post to me is laughably ignorant and directly contradicted by science:
"[LLMs] do not build models or reason about them"
-
What? I didn't even mention or refer to you in the text you quoted.
-
@devsimsek Hi,
I just wanted to ask what the difference between this and fitting a simple regression model on predicted outcomes? I feel the conclusions of this study is pretty obvious even from a simple regression case - or is it not? Why would we expect something different from LLMs?@drmambobob hi
Before I reply completely you must know that I'm not an expert on this topic but just a curious guy.You made a great remark, it is essentially the same problem scaled up. The obvi part is that you cannot get something from nothing. The main reason I assume the paper authors are talking about it differently with llms is that while a regression model just gets stale, olm goes probabilistic hell. It basically actively deletes its output diversity until it produces a single acceptable response.
I hope this answers your question.
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
-
Researchers just mathematically proved that AI can't recursively self-improve its way to superintelligence.
Not "we think it's unlikely." Not "it seems hard." Formally proved.
The model doesn't climb toward AGI — it slowly forgets what reality looks like. They call it model collapse. The math calls it inevitable.
I wrote about ithttps://smsk.dev/2026/04/26/ai-cannot-self-improve-and-math-behind-proves-it/
@devsimsek I am the farthest thing from a mathematician but just from a general lay persons perspective the thesis seems a little rigid. Assuming all training data outside the internet is corrupt, assuming that synthetic data only comes from recursive souring, etc. I don't really want to cheerlead AGI as I assume it will bring about a bunch of existential risk but I'm still in the "wouldn't bet against it" camp.
-
-
@devsimsek I am the farthest thing from a mathematician but just from a general lay persons perspective the thesis seems a little rigid. Assuming all training data outside the internet is corrupt, assuming that synthetic data only comes from recursive souring, etc. I don't really want to cheerlead AGI as I assume it will bring about a bunch of existential risk but I'm still in the "wouldn't bet against it" camp.
@mike Yeah I see your pov. I just made some remarks and some of them were satirical which you can obviously see through my writing style. I do agree with your stance on rigidity.
Also thanks for the comment
I hope you have enjoyed scrolling through those gifs -
@mike Yeah I see your pov. I just made some remarks and some of them were satirical which you can obviously see through my writing style. I do agree with your stance on rigidity.
Also thanks for the comment
I hope you have enjoyed scrolling through those gifs @devsimsek Forgive me as I digress. Even though I left Twitter years ago whenever I even mildly challenge someone on social media I still "brace for impact" after I press send. It's so refreshing to get a polite and coherent response that resembles civil discourse. Mastodon still amazes me.
-
@mike Yeah I see your pov. I just made some remarks and some of them were satirical which you can obviously see through my writing style. I do agree with your stance on rigidity.
Also thanks for the comment
I hope you have enjoyed scrolling through those gifs @devsimsek @mike I have a vague feeling that there's an argument to be made that a model that can train itself which is then given continuous IO from/to the world (vision, hearing, touch, voice, body) might be able to evolve into a self-sustaining intelligent entity, as long as it has to work for its survival (i.e. work for its input energy).
-
@devsimsek @mike I have a vague feeling that there's an argument to be made that a model that can train itself which is then given continuous IO from/to the world (vision, hearing, touch, voice, body) might be able to evolve into a self-sustaining intelligent entity, as long as it has to work for its survival (i.e. work for its input energy).
@virtuous_sloth @mike On a personal belief I don't think so but this must be tried or at least done an experiment.
I believe someone out there are currently working on it
.