Guess you have not tried more advanced prompting techniques like CoT, Agents and RAG.

Yeah, if an LLM was truly capable of reasoning, then whenever it makes a mistake, e.g. due to randomness or due to lack of knowledge, then pointing out the mistakes and giving steps on correcting the mistakes should result in basically a 100% success rate, since the assistant has infinite capacity to accommodate the LLM's weaknesses.

When you look at things like https://arxiv.org/abs/2408.06195 you notice that the amount of tokens needed to solve trivial tasks is somewhat ridiculous. On the order of 300k tokens for a simple grade school problem. That is roughly three hours at a rate of 30 token/s. You could fill 400 pages of a book with that many tokens.

> super-human knowledge but the reasoning skills of maybe a young child

Super-human knowledge is certainly true (all of wikipedia in multiple languages at all times, quickly)

Consider however, an important distinction. A young child is really, exactly not the way to think of these machines and their outputs. The implicit connection there is that there is some human-like progress to more capability.. not so.

Also note that "chain of reasoning" around 2019 or so, was exactly the emergent behavior that convinced many scientists that there was more going on that just a "stochastic response" machine. Some leading LLMs do have the ability to solve multi-step puzzles, against the expectations of many.

My "gut feeling" is that human intelligence is multi-layered and not understood; very flexible and connected in unexpected ways to others and the living world. These machines are not human brains at all. General Artificial Intelligence is not defined, and many have reasons to spin the topic in public media. Let's use good science skills while forming public opinion on these powerful and highly-hyped machines.

Thats true, for OpenAI's/FAIR's LLM's sure... Claude Opus or Sonnet 3.5? Not in my experience...

Empirically while you wait for yet another print on arxiv, I use the Sonnet 3.5 API every day to reason problems, iterate ideas and write highly optimised python and things like CUDA kernals. There is some degree of branching, guiding and correction; but oh boy there is certainly higher order synthesis and causal reasoning going on and its certainly not all from me.