1
point
viscanti
Joined 2,193 karma
I build things.
- viscanti parentI believe the author's thesis is that if they had invested in innovation over a couple decades, the product probably would have sucked less.
- They had pretty drastic price cuts on Opus 4.5. It's possible they're now selling inference at a loss to gain market share, or at least that their margins are much lower. Dario claims that all their previous models were profitable (even after accounting for research costs), but it's unclear that there's a path to keeping their previous margins and expanding revenue as fast or faster than their costs (each model has been substantially more expensive than the previous model).
- Well how much of it is correlation vs causation. Does the next generation of model unlock another 10x usage? Or was Claude 3 "good enough" that it got traction from early adopters and Claude 4 is "good enough" that it's getting a lot of mid/late adopters using it for this generation? Presumably competitors get better and at cheaper prices (Anthropic charges a premium per token currently) as well.
- It's more like the patient needs some fixed amount of food each day and it doesn't make a lot of sense to create lots more food than they need on the hopes that someday they'll want to eat more than they can.
If the argument is that everyone should focus on the arts at the expense of everything else, it's hard to imagine that's an ideal outcome relative to alternatives. If we're not arguing that everyone should focus 100% on the arts (no other degrees should be available), then it's a matter of degree and certainly some outcomes might end up with more people pursuing the arts than what society needs.
- If it's not astroturfing, the people who are so vocal about it act in a way that's nearly indistinguishable from it. I keep looking for concrete examples of use cases that show it's better, and everything seems to point back to "everyone is talking about it" or anecdotal examples that don't even provide any details about the problem that Gemini did well on and that other models all failed at.
- If you're able to generate minified code from all the code you can find on the internet, you end up with a very large training set. Of course in some scenarios you won't know what the original variable names were, but you would expect to be able to get something very usable out of it. These things, where you can deterministically generate new and useful training data, you would expect to be used.
- Because of how trivial that step is, it's likely pretty easy to just take lots of code and minify it. Then you have the training data you need to learn to generate full code from minified code. If your goal is to generate additional useful training data for your LLM, it could make sense to actually do that.
- > I can't understand the hate
I think it's because of the promises of the team (new Large Action Model) vs what's actually being delivered (the model is some scripts). The team has a history of over promising and underdelivering (or scamming - depending on your perspective). It's also economically unviable. Somehow you're meant to get free LLM calls for life but there's no way for them to actually cover those. There's not really any communication about how it might be a limited time thing for early adopters or how it could ever get to be sustainable.
If they had focused on what they have, they probably could have charged the same amount and people would generally be OK with it. But they've over promised and under delivered again. I think the reaction is pretty understandable.
- It seems to be difficult to turn the pure research back into new products. Apple famously got lots of ideas for free from Xerox PARC. Google researchers wrote the Attention Is All You Need paper and they're now desperately playing catchup because they couldn't convert it to any kind of product. There's nothing wrong with companies investing in pure research, but these large companies sometimes are unable to take advantage of the research. The people running the business want to keep doing what got them successful, not some new experimental thing that might not work.
- On device or in an Apple owned DC. It sounds like they have aspirations for their own Apple owned LLM. ChatGPT seems like it's there until they can get something good enough to generally replace it for cases where their in-house solution isn't capable enough yet. They likely continue to invest heavily on big capable LLMs as well as ones that are small enough to run on device (while working on the hardware side to ensure they have the device capabilities to run more powerful models on the device).
- Well the unit economics don't make any sense. You get unlimited free LLM calls forever? I don't really see the argument for it shipping too soon (although it is too soon for what they're claiming). There's a basic unit economics problem that can't be solved regardless of their future roadmap. They're promising people a no subscription model, claiming it eventually will have a Large Action Model and LLM for question and answering, and somehow those will be available free forever? It's either a get-rich-quick scheme, which sounds likely given their background as crypto grifters, or complete incompetence (somehow no one there ever thought what it might cost to do inference). Neither seems like a legit company that just shipped too soon.
- > I wonder if China or other countries will do the same for US apps now.
China has been doing this for a very long time, to keep US apps out of the Chinese market (or forced IP transfers to Chinese companies) so that Chinese apps can thrive. There's no room for them to retaliate because they've already gone as far as they possibly could.
- Zuck renamed the company Meta, partially because of a deep believe in the Metaverse (perhaps it's too early to call it, but it seems misguided) and partially because of massive amounts of negative press they were getting because of pretty toxic decisions that he was ultimately responsible for at Facebook. I love that he's passionate about AI and that he's supporting the release of somewhat open source models. But it seems a stretch to call him the best right now.
- > AI will make it far easier to keep people in line.
An alternative explanation is that AI companies are using RLHF to help the 90% understand that a lot of things aren't black and white but that there is relevant context to keep in mind. For many cases, that's probably useful (like if someone is asking if one movie is better than another movie). At the extremes, which are hopefully rare (are a lot of people really wondering if Hitler is better or worse than some Twitter posts), you get nonsense.
- > One drawback might be that political will and attention is fleeting; and interests become vested in a current state, blocking change; and it's now or not for another generation or so.
If a problem, like tracking what non SOTA models are doing with respect to safety, don't rise to being worth regulated, then why bother regulating it? If you're going to force everyone in the field, in California, to file additional paperwork (and understand what they're legally supposed to report and how and when), then you probably should have a compelling argument for why. If you didn't do that now, and it's never a big enough problem to regulate then you've done no real harm. If it's clearly an issue that needs to be regulated in the future then it should be easy to do so.