Maybe if instructed, but how would it know it needs to use python in this case vs just answer? Perhaps you'd instruct it to always attempt using code to reduce errors.

But the idea of trivial problems like this potentially causing issues for LLMs might mean other aspects of intelligence could also be a struggle for LLMs (which could impact it's coding ability as well).