- It's becoming challenging to really evaluate models.
The amount of intelligence that you can display within a single prompt, the riddles, the puzzles, they've all been solved or are mostly trivial to reasoners.
Now you have to drive a model for a few days to really get a decent understanding of how good it really is. In my experience, while Sonnet/Opus may not have always been leading on benchmarks, they have always *felt* the best to me, but it's hard to put into words why exactly I feel that way, but I can just feel it.
The way you can just feel when someone you're having a conversation with is deeply understanding you, somewhat understanding you, or maybe not understanding at all. But you don't have a quantifiable metric for this.
This is a strange, weird territory, and I don't know the path forward. We know we're definitely not at AGI.
And we know if you use these models for long-horizon tasks they fail at some point and just go off the rails.
I've tried using Codex with max reasoning for doing PRs and gotten laughable results too many times, but Codex with Max reasoning is apparently near-SOTA on code. And to be fair, Claude Code/Opus is also sometimes equally as bad at doing these types of "implement idea in big codebase, make changes too many files, still pass tests" type of tasks.
Is the solution that we start to evaluate LLMs on more long-horizon tasks? I think to some degree this was the spirit of SWE Verified right? But even that is being saturated now.
- I stopped listening to Lex Fridman after he tried to arbiter a "peace agreement" between Russia and Ukraine and claimed he just wanted to make the world "love" each other more.
Then I found out he was a fraud that had no academic connection to MIT other than working there as an IC.
- Mega cool, I’m curious if there’s a way to burn the ISO to a disc and get this playing on a physical console?
- I’ve felt the same. Also the AGI outcome for software engineers is:
A) In 5 years no real improvement, AI bubble pops, most of us are laid off. B) In 5 years near—AGI replaces most software engineers, most of us are laid off.
Woohoo. Lose-lose scenario! No matter how you imagine this AI bubble playing out, the musics going to stop eventually.
- This is such a good idea!
Kids music toys are often just purely toys tap a button, make a sound... But the skill ceiling could be so much higher, offering the ability to learn and express themselves more. Awesome work.
- So much text and not a single example, diagram, or demo.
I'm honestly skeptical this will work at all, the FOV of most webcams is so small that it can barely capture the shoulder of someone sitting beside me, let alone their eyes.
Then what you're basically looking for is callibration from the eye position / angle to the screen rectangle. You want to shoot a ray from each eye and see if they intersect with the laptop's screen.
This is challenging because most webcams are pretty low resolution, so each eyeball will probably be like ~20px. From these 20px, you need to estimate the eyeball->screen ray. And of course this varies with the screen size.
TLDR: Decent idea, but should've done some napkin math and or quick bounds checking first. Maybe a $5 privacy protector is better.
Here's an idea:
Maybe start by seeing if you can train a primary user gaze tracker first, how well you can get it with modeling and then calibration. Then once you've solved that problem, you can use that as your upper bound of expected performance, and transform the problem to detecting the gaze of people nearby instead of the primary user.
- Thought this was going to be a good article then the author started mentioning water consumption and I stopped reading.
- $30 for a sock even in 2025 seems pretty steep. In 2004 is crazy. I guess I'm forgetting how "overpriced" Apple was at the time.
- Here’s another idea:
We’ve had GPT2 since 2019, almost 6 years now. Even then, OpenAI was claiming it was too dangerous to release or whatever.
It’s been 6 years since the path started. We’ve gone from hundreds of thousands -> millions -> billions -> tens of billions -> now possibly trillions in infrastructure cost.
But the value created from it has not been proportional along the way. It’s lagging behind by a few orders of magnitude.
The biggest value add of AI is that it can now help software engineers write some greenfield code +40% faster, and help people save 30 seconds on a Google search -> reading a website.
This is valuable, but it’s not transformational.
The value returned has to be a lot higher than that to justify these astronomical infrastructure costs, and I think people are realizing that they’re not materializing and don’t see a path to them materializing.
- I was so confused by this article.
I was confusing it with TinyPilot, a hardware KVM made by an indie hacker Michael Lynch, that I think has since been acquired.
- Dunno if this passes the bootstrapping test.
This is sensitive to the initial candidate set of labels that the LLM generates.
Meaning if you ran this a few times over the same corpus, you’ll probably get different performance depending upon the order of the way you input the data and the classification tag the LLM ultimately decided upon.
Here’s an idea that is order invariant: embed first, take samples from clusters, and ask the LLM to label the 5 or so samples you’ve taken. The clusters are serving as soft candidate labels and the LLM turns them into actual interpretable explicit labels.
- This flavor of "FOMO" is new and tuned for capitalism/materialism, hence everyone wants to be go to trendy restaurants/travel/airbnb lifestyle.
But before "FOMO" used to be religious, more like "FOGTH", or fear of going to hell.
So people were mostly happy living simple, sweet lives. Spend time with family. Raise your children. Have a simple job. Just don't go to hell, so go to church, pray, don't sin. Everything will be great, as long as you don't go to hell.
Probably in like the 60s did consumerism become the mainstream religion and it's been taking over since. Now you *must* make more money to: buy a house, take a fancy vacation, live a luxurious retirement, etc.
The cringe "hustle culture" of today is because for some people, it's their spiritual fulfillment. It is their religion. Their main focal point of existence is to buy bigger, buy better. It's almost taboo to consider an early retirement, "omg, I'd get so bored! I'd go crazy!"
How dare you not follow my religion of selling B2B SaaS? You are not a go-getter, and I am. Did I mention I also have a podcast?
- On paper, nothing will happen. The wealthy in America will continue to get wealthier. Real estate and stocks will soar in value.
In reality, the dollar's true value will plummet. The FED is starting to lower interest rates again. We are likely going to undergo brutal inflation.
Crashing the economy is obviously very politically unpopular. The left/right will do whatever they can to keep this charade up, even if it means dooming the working class and throwing them some kind of bone to make them think they're ok.
The COVID pandemic was a good example of this. The working class got thrown a $2,000 check while there was billions given to bail out businesses/lots of fraud. Not a lot of people cared because hey, we got a $2k check... Even though that $2k check was not even close to maintaining their relative wealth pre-pandemic due to all of the government's inflationary measures.
There won't be a recession, it won't happen on paper. But the middle/working class will continue to be squeezed. And there will be programs to "rescue us." Maybe it's low cost home programs, maybe it's community college, I'm not sure. But I am sure it will never truly benefit the working/middle class, it'll just be a token to keep them from fully dying.
- GPUs still won't be cheap
- Almost every single one of these "using WiFi to do X! Crazy!" articles always requires some crazy amount of calibration / training requirement.
It's like me telling you: With using just audio I can trace your exact coordinates! ... By using an array of microphones in a room.
- Thanks, I googled "bald suicide risk pubmed" and it was the first article I found. I can't find any with as clear-cut statistics for AA.
I still think it applies though; the psychological effect of being bald probably doesn't care much about the underlying cause.
- I take dutasteride (even stronger form of finasteride) and have been personally fortunate to not have any side effects in ~8 years.
My whole family on both sides is bald bald. Without it I would have been bald bald in my early 20s.
I knew there were risks of hormone disruption/mood swings/etc but I still prefer it over being bald. I took a calculated risk and I'm thankful that the drug exists. Please don't prosecute the people that gave me the only real, working treatment for my baldness.
- Not kidding: What about suicides prevented from not going bald? To paint a fair image, this study should also compare suicide prevalence for bald people.
-------
Results: Compared with the controls, an increased risk of suicide attempts was observed in patients with AA, with an adjusted hazard ratio of 6.28 (95% confidence interval, 4.47-8.81). Suicide risk remained significantly elevated in AA patients when stratified by underlying psychiatric disorders. The mean age of initial suicidal behaviors was also lower in patients with AA.
Conclusions: Patients with AA had a significantly higher incidence of suicidal attempts than controls, regardless of concurrent psychiatric illness. Further studies are needed to elucidate the pathophysiology of the association between AA and suicidality. In addition, dermatologists should be aware of the increased suicidality of patients with AA.
https://pubmed.ncbi.nlm.nih.gov/36921592/
----
Not enough time to size these in comparable numbers to this study, but would be really interesting.
Is it a net positive (reduced risk) over just going bald?
- My wife (then girlfriend) and I were at a concert. She went to the bathroom to pee. She came back crying.
I told her she might have a UTI. It was not normal for it to hurt that badly to pee. She denied it. I bought her a UTI test, it came out positive. She was shaking. I told her we had to go to the hospital, she thought they were period cramps.
I call a teledoc. They video chat. She explains the pain shes feeling in her lower back means it’s likely a UTI, the infection has likely reached her kidneys, and we should go to the ER immediately.
In the ER we think they’re going to just give her some antibiotics and send her home. Nope. She throws up. Things go bad fast. Her heart rate is 160. She turns a color I’ve never seen a human before.
The next 3 days were so incredibly hard. But I’m so thankful to all the medical workers that were attentive to us.
Thankfully she makes a full recovery. For a week or so she was lethargic/tired but she’s fully healthy now.
A few months before I had read a story about a woman who’s boyfriend had died from a UTI because they went to a gospital, gave him some antibiotics, and he ending up dying at home because the infection was already too progressed to fight off at home.
Had the person who evaluated my girlfriend not evaluated seriously or just sent her off that could’ve been her. I’m so thankful they admitted her and took her care seriously.
It’s scary how quickly a UTI or some other benign infection can become sepsis. Take it seriously.
"Bring me the rats."