- Scene_Cast2If you're into carrier landings and have a VR headset, I highly recommend checking out VTOL VR.
- I'd guess that the datacenter "GPUs" lack all the fixed-function graphics hardware (texture samplers, etc) that's still there in modern consumer GPUs.
- There is friction to asking AI yourself. And a comment typically means that "I found the AI answer insightful enough to share".
- Ah, I think I agree. There could be a potential unrelated handicap, so there is a lack of a guarantee or a proof.
- With what backhaul? WiFi?
- I'm not sure how likely it is that an answer would fall outside of the top-p of 0.95 (used in the paper). A random number generator would also need an unreasonably high number of samples to get a correct answer. I think figures 17 and 18 are interesting for this discussion too, they show performance at various sampling temperatures. I think the point of the paper is that RL "sharpens" the distribution of non-RL nets, but it does not uncover any new reasoning paths - non-RL nets already had multiple decently high probability paths of answering questions to begin with, and RL reuses a subset of those.
- I think my favorite of the bunch is the "Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model" paper. Easy to read, gets the point across very intuitively and quickly, and the point is very interesting and relevant to a lot of people.
About the Superposition paper - this is close to what I've been thinking about over the past week. I'm thinking that concepts or choices in a "superposition" are harder for a fully-differentiable neural net to reason about. For example, if there's a "green" vs "purple" choice to be made, it can't fully commit to either (especially if they're 50-50), and will have to reason about both simultaneously (difficult due to nonlinear manifold space). Discretizing to tokens (non-differentiable argmax) forces a choice, and that allows it to reason about a single concept separately and easier.
- Most likely the op amp (or whatever gain stage) noise. After a certain point, you get thermal noise.
But with such scales, low sample rates and averaging are key.
- This post assumes C/C++ style business logic code.
Anything HPC will benefit from thinking about how things map onto hardware (or, in case of SQL, onto data structures).
I think way too few people use profilers. If your code is slow, profiling is the first tool you should reach for. Unfortunately, the state of profiling tools outside of NSight and Visual Studio (non-Code) is pretty disappointing.
- I found that taking a specific brand of Vitamin D (the Genestra D-mulsion in particular) right before bed was guaranteed to give me vivid dreams. I've had half a dozen friends try it, with every single one reporting similar results.
- Yep. MoE, FlashAttention, or sparse retrieval architectures for example.
- Still way pricier (>2x) than Gemini 3 and Grok 4. I've noticed that the latter two also perform better than Opus 4, so I've stopped using Opus.
- For anyone looking for some IDEs to tinker around with shaders:
* shadertoy - in-browser, the most popular and easiest to get started with
* Shadron - my personal preference due to ease of use and high capability, but a bit niche
* SHADERed - the UX can take a bit of getting used to, but it gets the job done
* KodeLife - heard of it, never tried it
- Matmuls (and GEMM) are a hardware-friendly way to stuff a lot of FLOPS into an operation. They also happen to be really useful as a constant-step discrete version of applying a mapping to a 1D scalar field.
I've mentioned it before, but I'd love for sparse operations to be more widespread in HPC hardware and software.
- My biggest worry is that it's harder and harder to find a phone with an unlockable bootloader.
- Super cool. Also, this is an example of why having an open OS is awesome.
- I've noticed that image models are particularly bad at modifying popular concepts in novel ways (way worse "generalization" than what I observe in language models).
- I would love to hear your (and others) opinions.
I don't have a good idea of what happened inside or what they could have done differently, but I do remember them going from a world-leading LLM AI lab to selling embeddings to enterprise.
- GPU horsepower isn't there for that.
- I swapped all my AA and AAA batteries for Enelooops. The cheaper white ones are the best for most applications.