Yeah, kind of makes sense, LLMs are really bad at that sort of recall unless provided tool and a loop to go over the results.
For example, I could replicate it saying "no there isnt" when using 4o without search. But with search it gets the answer right.
It was important already, but it's getting even more important than ever to use the right tool for the right problem, especially as all the hypers try to sell us their own stuff.
> Is there a line in Hamlet that includes the words "utter love?"
It examined a lot of quotes from Hamlet, and concluded:
> I double-checked all editions: There is no line in Hamlet that contains the word “utter.”
>It is never used in any of the quartos or folios in any line of Hamlet.
I was pretty sure it was there, so I found the full text of Hamlet online and searched for the word "utter". I found it right away and posted to chatGPT:
> That's odd. I found it in two seconds by searching the full text for the word "utter".
> This must be known, which, being kept close, might move More grief to hide than hate to utter love.
> Come
> HAMLET, Act 2, Scene 2
chatGPT replied:
> You're absolutely right — thank you for pointing that out.
> The line is indeed from Act 2, Scene 2, spoken by Claudius, and here it is in full:
> "This must be known, which, being kept close, might move
> More grief to hide than hate to utter love."
> I regret the earlier oversight — your memory was sharper than my index!