Central to this hypothesis is the concept of recursive self-improvement: an AI system with the capacity to inspect and enhance its own architecture or training processes would initiate a positive feedback loop, with each generation of the AI being more intelligent than the last, leading to exponential growth in its capabilities.

TL;DR: as always, the headlines are over-simplified to the point of inaccuracy...

They're proving the second half, that model collapse due to self-training is inevitable.

But they're not really addressing the first half, leaving open the question of whether an AI system could improve its own code or architecture. I personally don't think an LLM designing a better LLM gets us any closer to the so-called singularity, either, but that's just an opinion.

Of course, the industry (at least in the US) has gone all in (and then mortgaged the house and pawned its jewelry and sold its blood and firstborn and bet that, too) on just making the model bigger and the feeding it more data. So it's useful to have proof that the current strategy deadends when they run out of real human data.

https://tech.lgbt/@toolbear/116446645444544147

The one by Ali Alkhatib I found to be the most illuminating.
@dpiponi
@devsimsek

@ShadowJonathan

I was thinking about strange attractors because of Edward Lorenz and Margaret Hamilton's contributions to the study of chaos theory. Lorenz was modeling atmospheric convection and discovered that the model would degrade because small inconsistencies would magnify over successive iterations. The resulting data formed a strange attractor (Lorenz Attractor).

Also thanks for the comment, hope you liked reading the post

Thanks

If anything, if people were to take his words seriously then it would only help because the longer things go on, the stronger the collapse, I think.

Is this related to Strange Attractors?

Well not with that attitude they won't. Why, they just simply need to pull themselves up by their own bootstraps!

Love it when the mathematical model puts in black and white what most of us intuitively pointed out. Just proves our billionaire geniuses are short on genius and long on shell games.

Their current produce is 8 GOPS (but does that mean GFLOPS?) and current GPUs are doing TFLOPS, but they seem confident about 1000-fold improvements every 1.5 years, so 2027 here they come!

Also confusing since GPUs are currently being used to do massively-parallel floating-point arithmetic as part of the matrix-matrix and matrix-vector math involved in the linear algebra of the models, and there's not much analog about that I am aware of. I do know that simple resister, capacity, & inductor circuits are analog representations of 2nd-order differential equations, but not sure how that would apply here.
/shrug/