"emergent superintelligent AI" is as much superstition as believing in imaterial souls. One company literally used the term "people spirits" to refer to how LLMs behave in their official communications.
It's a cult. Like many cults, it tries to latch on science to give itself legitimacy. In this case, mathematics. It has happened before many times.
You're trying to say that, because it's computers and stuff, it's science and therefore based on reason. Well, it's not. It's just a bunch of non sequitur.
I didn't say anything about "emergent superintelligent AI".
I'm confused.
We are on a comment section about a post with AGI in the title.
The term is scientifically vague, but it is estabilished in the popular culture that it is related to superintelligence and emerging behavior. If you don't agree, you owe the reader a better definition.
Given this context, if you're not talking about that, what are you talking about then?
(Aside from "explaining" why AI couldn't ever possibly be "really intelligent" for those who find this notion existentially offensive.)