My context window is about a day. I can remember what I had for lunch today, and sometimes what I had for lunch yesterday. Beyond that, my lunches are gone from my context window and are only in my training data. I have vague ideas about what dishes I ate, but don't remember what days specifically. If I had to tell you what separate dishes I ate in the same meal, I don't have specific memories of that. I remember I ate fried plantains, and I ate beans & rice. I assume they were on the same day because they are from the same cuisine, and am confident enough that I would bet money on it, but I don't know for certain.
One of my earliest memories is of painting a ceramic mug when I was about 3 years old. The only reason I remember it is because every now and then I think about what my earliest memory is, and then I refresh my memory of it. I used to remember a few other things from when I was slightly older, but no longer do, because I haven't had reasons to think of them.
I don't think humans have specific black and white differences between types of knowledge that way LLMs do, but there is definitely a lot of behavior that is similar to context window vs training data (and a gradient in between). We remember recent things a lot better than less recent things. The quantity of stuff we can remember in our "working memory" is approximately finite. If you try to hold a complex thought in your mind, you can probably do that indefinitely, but if you then try to hold a second equally complex thought as well, you'll often lose the details of the first thought and need to reread or rederive those details.
One of my earliest memories is of painting a ceramic mug when I was about 3 years old. The only reason I remember it is because every now and then I think about what my earliest memory is, and then I refresh my memory of it. I used to remember a few other things from when I was slightly older, but no longer do, because I haven't had reasons to think of them.
I don't think humans have specific black and white differences between types of knowledge that way LLMs do, but there is definitely a lot of behavior that is similar to context window vs training data (and a gradient in between). We remember recent things a lot better than less recent things. The quantity of stuff we can remember in our "working memory" is approximately finite. If you try to hold a complex thought in your mind, you can probably do that indefinitely, but if you then try to hold a second equally complex thought as well, you'll often lose the details of the first thought and need to reread or rederive those details.