jacobgkau parent
Maybe they should be trained on the understanding that making up a source is not "doing what you ask of them" when you ask for a source. It's actually the exact opposite of the "doing what you asked, not what you wanted" trope-- it's providing something it thinks you want instead of providing what you asked for (or being honest/erroring out that it can't).
Think for a second about what that means... this is a very easy thing to do IFF we already had a general purpose intelligence.
How do you make an LLM understand that it must only give factual sources? Just some naive RL with positive reward on the correct sources and negative reward on incorrect sources is not enough -- there are obscenely many more hallucinated sources possible, and the set of correct sources is a set of insanely tiny measure.
"Easy". You make the model distinguish between information and references to information. Information may be fabricated (for example, a fictional book is mostly composed of lies) but references are assumed to be factual (a link does point to something and is related to something). Factual information is true only to the degree that it is conveyed exactly, so the model needs to be able to store and reproduce references verbatim.
Of course, "easy" is in quotes because none of this is easy. It's just easier than AGI.