Preferences

Guess: Some human saw the q & a. They realized it wasn't good. They uploaded some example of a phrase and the meaning, which would fix it. They were kinda lazy and just fixed that specific scenario.

Doesn't that still call into question the quality of GPT-3? Surely such a large model should be able to extrapolate to "which is better: a or b?" from "is a better than b?" when only provided with the latter.
There is no definitive answer to this question, as it depends on the situation.
It doesn’t call it into question for me, but perhaps I just had lower expectations to start with.

I forget where I saw this comparison so I can’t link to it, but the last few years in AI are like waking up and finding dogs can talk: while some complain they’re not the worlds greatest orators, I find it amazing they can string a few genuinely coherent sentences together and maintain a contextual thread over multiple responses even half the time.

Perhaps you are thinking of Scott Aaronson's "AlphaCode as a dog speaking mediocre English"? [1]

I agree with the sentiment, but to continue your analogy, if OpenAI is using people to improve the answers to specific questions, it is a bit like learning that Cicero, Lincoln and Churchill were merely reading the work of speechwriters.

There is an argument that it does not matter how GPT-3 gets to its answers - after all, for a long time, the main approach to AI was for people to write a lot of bespoke rules in an attempt to endow a computer with common sense and knowledge, so GPT-3 + instructGPT might be described as a hybrid of machine learning and the old approach.

If OpenAI wishes to pursue that path, it is fine by me (as if my opinion matters!) but, because the perception of GPT-3 depends very strongly on how its output looks to human readers, it is obviously misleading if some of the most impressive replies were largely the result of specific human intervention. The issue is transparency: I would just like to know, when I read a reply, if this was the case, and it would not help OpenAI for it to ignore the call, in this article, for it to be clear about this.

There is another argument that says that, given how GPT-3 works, it is unreasonable to expect it to give good answers in these cases - but that's the point! It looks really impressive when GPT-3 apparently does so, but not if they were effectively hard-coded.

[1] https://scottaaronson.blog/?p=6288

Thanks for the link, I was either thinking of that or someone who was referencing that.
Perhaps, I'm not sure. I guess it depends on what's causing it to stumble.

This item has no comments currently.

Keyboard Shortcuts

Story Lists

j
Next story
k
Previous story
Shift+j
Last story
Shift+k
First story
o Enter
Go to story URL
c
Go to comments
u
Go to author

Navigation

Shift+t
Go to top stories
Shift+n
Go to new stories
Shift+b
Go to best stories
Shift+a
Go to Ask HN
Shift+s
Go to Show HN

Miscellaneous

?
Show this modal