- > you would have never posted that on an article about alcohol
Well of course not, as the two drugs have completely different intoxication side effects.
- > There are so many theories regarding human cognition that you can certainly find something that is close to "autocomplete"
Yes, you can draw interesting parallels between anything when you're motivated to do so. My point is that this isn't parsimonious reasoning, it's working backwards from a conclusion and searching for every opportunity to fit the available evidence into a narrative that supports it.
> Roots of predictive coding theory extend back to 1860s.
This is just another example of metaphorical parallels overstating meaningful connections. Just because next-token-prediction and predictive coding have the word "predict" in common doesn't mean the two are at all related in any practical sense.
- There's no evidence for it, nor any explanation for why it should be the case from a biological perspective. Tokens are an artifact of computer science that have no reason to exist inside humans. Human minds don't need a discrete dictionary of reality in order to model it.
Prior to LLMs, there was never any suggestion that thoughts work like autocomplete, but now people are working backwards from that conclusion based on metaphorical parallels.
- > First: a selection mechanism is just a selection mechanism, and it shouldn't confuse the observation of an emergent, tangential capabilities.
Invoking terms like "selection mechanism" is begging the question because it implicitly likens next-token-prediction training to natural selection, but in reality the two are so fundamentally different that the analogy only has metaphorical meaning. Even at a conceptual level, gradient descent gradually honing in on a known target is comically trivial compared to the blind filter of natural selection sorting out the chaos of chemical biology. It's like comparing legos to DNA.
> Second: modern models also under go a ton of post-training now. RLHF, mechanized fine-tuning on specific use cases, etc etc. It's just not correct that token-prediction loss function is "the whole thing".
RL is still token prediction, it's just a technique for adjusting the weights to align with predictions that you can't model a loss function for in per-training. When RL rewards good output, it's increasing the statistical strength of the model for an arbitrary purpose, but ultimately what is achieved is still a brute force quadratic lookup for every token in the context.
- Reinforcement learning is a technique for adjusting weights, but it does not alter the architecture of the model. No matter how much RL you do, you still retain all the fundamental limitations of next-token prediction (e.g. context exhaustion, hallucinations, prompt injection vulnerability etc)
- I'm not exactly sure what you mean. Could you please elaborate further?
- Not at all. Why would it?
- > Don’t let some factoid about how they are pretrained on autocomplete-like next token prediction fool you into thinking you understand what is going on in that trillion parameter neural network.
This is just an appeal to complexity, not a rebuttal to the critique of likening an LLM to a human brain.
> they are not “autocomplete on steroids” anymore either.
Yes, they are. The steroids are just even more powerful. By refining training data quality, increasing parameter size, and increasing context length we can squeeze more utility out of LLMs than ever before, but ultimately, Opus 4.5 is the same thing as GPT2, it's only that coherence lasts a few pages rather than a few sentences.
- > it’s unclear what is their endgame here
Marketing. That defines pretty much everything Anthropic does beyond frontier model training. They're the same people producing sensationalized research headlines about LLMs trying to blackmail folks in order to prevent being deleted.
- What evidence do you have that the "MSM" are "carefully avoid mentioning" it?
- > The old company wasn't a domestic political adversary
That depends entirely on the politics in question. It's well known that corporations are willing to abuse their power for political ends if it serves their interests to do so.
- > However, the new company could say "turn on microphone for all vacuums in the DC area and send transcripts to us"
The old company could have done the same thing. I recognize that China is a u.s. geopolitical adversary, but when it comes to politics domestic adversaries are just as ruthless.
- > The general idea is that whenever algorithms are deciding what you see Section 230 is not in play
This isn't correct. The ruling was very narrow, with a key component being that a death was directly attributed to a trend recommend by the algorithm that TikTok was aware of, and knew was dangerous. That part is key - from a section 230 enforcement perspective it's basically the equivalent of not acting to remove illegal content. Basically everything we've understood about how algorithms are liable since section 230 was enacted remain intact.
- I don't think the comparison is valid. Releasing code and weights for an architecture that is widely known is a lot different than releasing research about an architecture that could mitigate fundamental problems that are common to all LLM products.
- As far as I'm aware, transitive dependencies are counted in this number. So when you npm install next.js, the download count for everything in its dependency tree gets incremented.
Beyond that, I think there is good reason to believe that the number is inflated due to automated downloads from things like CI pipelines, where hundreds or thousands of downloads might only represent a single instance in the wild.
- Optimizing for disk space is very low on the priority list for pretty much every game, and this makes sense since its very low on the list of customer concerns relative to things like in-game performance, net code, tweaking game mechanics and balancing etc.
- Every style of interview will cause anxiety, that's just a common denominator for interviews.
- Seems to me the root problem here is poor security posture from the package maintainers. We need to start including information about publisher chain of custody into package meta data, that way we can recursively audit packages that don't have a secure deployment process.
- How so?
The fact that legalization did not impact the crash rates is also a strong signal that THC itself is not causing the crashes.
The presence of THC in the blood is not a reliable signal for intoxication, so further research is needed to draw any type of conclusion.
Finally, it's also been noted that there are some sample bias concerns because the data comes from fatal crashes where it was determined that a drug test should be administered after the crash.