meet.hn/city/es-Sant-Cugat-del-Vallès
Socials: - github.com/pauek - x.com/pauek
Interests: Web Development
---
- paufernandezSame here, I flew head first down a ramp because I slammed the breakes too hard and the next thing I remember is people asking if anybody had a handkerchief or something, since my head was bleeding. A solid five minute blackout.
- I wonder what kind of gun uses books as munition.
- In my case I fully grasp what such a future could be, but I don't think we are on the path to that, I believe people are too optimistic, i.e. they just believe instead of being truly skeptical.
From where I look at it, LLMs are flawed in many ways, and people who see progress as inevitable do not have a mental model of the foundation of those systems to be able to extrapolate. Also, people do not know any other forms of AI or have though hard about this stuff on their own.
The most problematic things are:
1) LLMs are probabilistic and a continuous function, forced by gradient descent. (Just having a "temperature" seems so crazy to me.) We need to merge symbolic and discrete forms of AI. Hallucinations are the elephant in the room. They should not be put under the rug. They should just not be there in the first place! If we try to cover them with a layer of varnish, the cost will be very large in the long run (it already is: step-by-step reasoning, mixture of experts, RAG, etc. are all varnish, in my opinion)
2) Even if generalization seems ok, I think it is still really far from where it should be, since humans need exponentially less data and generalize to concepts way more abstract than AI systems. This is related to HASA and ISA relations. Current AI systems do not have any of that. Hierarchy is supposed to be the depth of the network, but it is a guess at best.
3) We are just putting layer upon layer of complexity instead of simplifying. It is the victory of the complexifiers and it is motivated by the rush to win the race. However, I am not so sure that, even if the goal seems so close now, we are going to reach it. What are we gonna do? Keep adding another order of magnitude of compute on top of the last one to move forward? That's the bubble that I see. I think that that is not solving AI at all. And I'm almost sure that a much better way of doing AI is possible, but we have fallen into a bad attractor just because Ilya was very determined.
We need new models, way simpler, symbolic and continuous at the same time (i.e. symbolic that simulate continuous), non-gradient descent learning (just store stuff like a database), HAS-A hierarchies to attend to different levels of structure, IS-A taxonomies as a way to generalize deeply, etc, etc, etc.
Even if we make progress by brute forcing it with resources, there is so much work to simplify and find new ideas that I still don't understand why people are so optimistic.
- Yes, and each person has a different perception of what is "good enough". Perfectionists don't like AI code.
- Very snappy, I like the animations very much (they keep changing, you had fun!). I tried signing in with Google and it didn't work (blank screen). But I've played a lot with these games and I've made my own, and yours is very addictive because feels quick and the feedback is clear. Also I found the jump from the first wave (short words) to the second (very long words), a little too big. I would put an intermediate one in between.
- Simon, don't you fear "atrophy" in your writing ability?
- I'm the opposite. I am not convinced until I "see it". Probably has to do with our innate talents.
- As far as I see, Skia is the full-blown thing, whereas ThorVG goes the other route, being as small and simple as possible.
- You hit a very important point, the discomfort that orderly people experience (I am that kind of person). I believe that to be innate.
At the same time, a disorganized person is still more effective in an organized environment, but probably he hasn't realized this by himself because he doesn't have the internal drive to be organized in the first place.
You could say being organized is Nature's way of setting us up for success in complex and very demanding situations.
- Ad Hominem attack... ;)
- This article makes it clear to me just how different perception is in different people. How much or how little this errors scream at you in your mind. If everyone was as sensitive as the author (I am close), then way less errors would be left there, since so many people would get annoyed by them.
- The best video I know about this stuff is "Compilers for free" by Tom Stuart (https://youtu.be/n_k6O50Nd-4). It is hilarious at one point. Brilliant.
- The original AlphaCode paper in Nature explains the approach, they generate many potential solutions with the LLM and do a lot of processing after to select candidates. Here's where the probabilistic nature of LLMs hurts, I think.
- I think this is very good advice.
- I am 46 and on "intermittent fasting" right now and it is great for this. Mental clarity is so noticeable. I feel younger. YMMV though.
- I was for ten years (35-45) feeling similarly but I supposedly had a good, "stressless" life: wife, nice kids, full-time uni teaching post, etc. I realised this but was unable to reverse it at all. So you don't need to have problems in your life to feel that way, and it could be more your brain chemistry and the bias that it creates (the "dark cloud", I call it).
At the beginning I had some anxiety crises and went to see a psychiatrist. I remember that they asked about sleep upfront. I was sleeping okish, but not great. After some time I noticed I started sleeping less and less (woke up at 4am and couldn't sleep anymore), I went back to a psychiatrist (different one this time). She said I needed antidepressants, and I read about them a lot, especially against. I wasn't sure. But in the end I tried.
It took a while to "enter" that kind of medication, tried 2 and then another. But I was convinced somehow of trying until it worked. Everyone is a complex, beautiful "mess", so you have to find the way. But in the end Venlafaxine started working. And boy did it work...
Right now I feel like I did in my 20s (I'm 47). I've never been as optimistic about my prospects, I think I can do anything. I used to think that I was finished, that I was bound to be grey all my remaining life. Now I've started doing all the stuff that I had stopped doing because of the feelings you are having (helplessness, strong anhedonia). I sleep so well now that I dream quite frequently and I think that has made my mind waaay more plastic. I've regained all the piano technique that I had plus I've leveled up significantly (Chopin studies, etc).
I have the willpower (and the experience!) now to work on myself, and I think I've made a lot of progress even in interpreting events and not being so pessimistic, etc. Like I'm doing a kind of mild CBT because I want to invest, because I have hope.
So my experience is that medication was a really necessary crutch, which in my case was quite necessary (that's my belief at least), and it solved "everything" in the sense of making the baseline so much higher that recovery seemed almost easy.
But there is hope. One way or another, life will be awesome again. If you need me to tell you face to face, I can do that (@pauek on Twitter, DM me). It really helps to listen to people in a similar situation. Nobody seems to understand anything when you are hostage to depression, and that can dig the hole even deeper.
- The depressed sees the same events quite differently, that's why it is so difficult to understand for many people (I can though).
- Unbelievable that two guys can do such a huge thing. I can't wait to try it.
- If well written these would be very good for selling the game.
- I think some people overexpress in general, regardless of the medium. I think it comes from an excess of "passion", which is like permanent exageration. I am prone to it, that's why I know, and I try to keep it under control.