If the Church-Turing thesis can be proven false, conversely, then it may be possible that such a program can't exist - it is a necessary but not sufficient condition for the Church-Turing thesis to be false.
Given we have no evidence to suggest the Church-Turing thesis to be false, or for it to be possible for it to be false, the burden falls on those making the utterly extraordinary claim that they can't exist to actually provide evidence for those claims.
Can you prove the Church-Turing thesis false? Or even give a suggestion of what a function that might be computable but not Turing computable would look like?
Keep in mind that explaining how to compute a function step by step would need to contain at least one step that can't be explain in a way that allows the step to be computable by a Turing machine, or the explanation itself would instantly disprove your claim.
The very notion is so extraordinary as to require truly extraordinary proof and there is none.
A single example of a function that is not Turing computable that human intelligence can compute should be low burden if we can exceed the Turing computable.
Where are the examples?
Doesn't that assume that the brain is a Turing machine or equivalent to one? My understanding is that the exact nature of the brain and how it relates to the mind is still an open question.
If the Church-Turing thesis is true, then the brain is a Turing machine / Turing equivalent.
And so, assuming Church-Turing is true, then the existence of the brain is proof of the possibility of AGI, because any Turing machine can simulate any other Turing machine (possibly too slowly to be practical, but it denies its impossibility).
And so, any proof that AGI is "mathematically impossible" as the title claims, is inherently going to contain within it a proof that the Church-Turing thesis is false.
In which case there should be at least one example of a function a human brain can compute that a Turing machine can't.
This is simply technically not true. You can look up the https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis and it does not talk about brains, or intelligence.
See https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#V... and https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#P...
None of us exist in a vacuum*, we all react to things around us, and this is how we come to ask questions such as those that led Gödel to the incompleteness theorems.
On the other hand, for "can a program prove it?", this might? I don't know enough Lean (or this level of formal mathematics) myself to tell if this is a complete proof or a WIP: https://github.com/FormalizedFormalLogic/Incompleteness/blob...
* unless we're Boltzmann brains, in which case we have probably hallucinated the existence of the question in addition to all evidence leading to our answer
Can you prove such a program may exist?