- There are enormous microcode, firmware and drivers blobs everywhere on any pathway. Even with very privileged access of someone at Intel or NVIDIA, ability to have a reasonable level of deterministic control of systems that involve CPU/GPU/LAN were long gone, almost for a decade now.
- The same argument was there about needing to be an expert programmer in assembly language to use C, and then same for C and Python, and then Python and CUDA, and then Theano/Tensorflow/Pytorch.
And yet here we are, able to talk to a computer, that writes Pytorch code that orchestrates the complexity below it. And even talks back coherently sometimes.
- > But as estrogen levels shift in perimenopause and beyond, this intense drive to please and nurture others begins to diminish. What replaces it isn’t bitterness. It’s clarity.
It's not clear how anxiety, mood swings, brain fog, inability to remember faces, fear, aggression are somehow being called "clarity".
- <think>Ok, the user is claiming that... </think> ....
- They are likely making informed decisions. Disney/Pixar are players, not bystanders.
- Can you give an example of some part of the physical world or infosphere that an LLM can't model, at least approximately?
- It's funny how Men vs Women in Science, or IQ standard deviation differences still result in such heated debates, while at the same time...
- Below is the worst quote... It is plain wrong to see an LLM as a bags of words. LLMs pre-trained on large datasets of text are world models. LLMs post-trained with RL are RL-agents that use these modeling capabilities.
> We are in dire need of a better metaphor. Here’s my suggestion: instead of seeing AI as a sort of silicon homunculus, we should see it as a bag of words.
- There are a lot of people around who are playing victims to get benefits. And relentlessly pushing their agenda.
- Kids glued to "fun" (addictive) video games on their phones is not wealth, it is the feeder channel for these "efficient" health "care" schedulers, in your other example.
- When I was growing up 90-ies, a mix of using public metro and buses to roam the city (since I was in second grade, when I was allowed to take metro to do afterschool karate) and spending summers in various countryside locations where my grandparents resided was a good mix.
I disagree that the kids need or want to roam without grownups all the time. Grownups are not the problem. Kids are fun for the parents, the company of parents and their peers is kinda amazing.
Systems and institutions are the problem. When kids are stuck in the daycare or school, in a very limited space, grownups are stuck at the office and grandparents are in a different state for tax purposes - that is the problem.
I don't know if this is true, but Patagonia claimed at some point that they used to maintain daycares and allow kids to roam the campus...
- It feels like to really censor the model it needs to be pre-trained on a distribution of data derived from a well defined and synthetic source, like TinyStories. Otherwise... world model would still be capable of modeling the original distribution.
- What if it is a 99% chance of being transformational and the results of that transformation are completely unpredictable?
- >> Heaven exists you just have to do this work to get there.
Or perhaps Karpathy has a higher level understanding and can see a bigger picture?
You've said something about heaven. Are you able to understand this statement, for example: "Heaven is a memeplex, it exists." ?
- Two decades ago, in the Bay Area we used to have a lot of books stores, specialized, chains, children's, grade school, college slugbooks, etc. Places like Fry's had a coffee and a book store inside. The population grew, number of book stores went down to near zero.
- Yes, this is true. There used to be a lot of local book stores, for example. Amazon optimized that away, while ruining social fabric.
- In that case, it's great that Microsoft is building their silicon. Keeps NVIDIA in check, otherwise these profits would evaporate into nonsense and NVIDIA would lose the AI industry to competition from China. Which, depending if AGI/ASI is possible or not, may or may not be a great move.
- safety/security patches
The result is interesting. First, juniors are miserable. What used to be a good experience coding and debugging, in a state of flow is now anxiously waiting if an AI could do it or not.
And senior devs are also miserable, getting apprentices used to be fun and profit, working with someone young is uplifting, and now it is gone.
The code quality is going down, Zen cycle interrupted, with the RL cost functions now at the top.
The only ones who are happy are hapless PhDs ;$