Really, I recommend reading this part of the thread while thinking about the analogy. It's great.
Competition fixes some of this, I hope Anthropic and Mistral are not far behind.
Just like employing other people!
"I can only ask my employee 20 smart things this week for $20?! And they get dumber (gpt-4o) after that? Not worth it!"
No. This may be common in freelance contracts, but is almost never the case in employment contracts, which specify a time-based compensation (usually either per hour or per month).
The only bit about it that feels at all truthful is this bit, which is glossed over but likely the only real factor in the decision:
> after weighing multiple factors including ... competitive advantage ... we have decided not to show the raw chains of thought to users.
I don't see why this is qualitatively different from a cost perspective than using CoT prompting on existing models.
If you set a limit, once it's hit you just get a failed request with no introspection on where and why CoT went off the rails
Ultimately if the output of the model is not worth what you end up paying for it then great, I don't see why it really matters to you whether OpenAI is lying about token counts or not.
I wouldn’t just implicitly trust a vendor when they say “yeah we’re just going to charge you for what we feel like when we feel like. You can trust us.”