- > using imagenet-1k for pretraining
Lecun still can't show JEPA competitive at scale with autoregressive LLM.
- AMD Pensando DPU
- Write a dockerfile and pay for a PaaS service.
- > Before too long (and we already start to see this) humanoid robots will get wheels for feet, at first two, and later maybe more, with nothing that any longer really resembles human legs in gross form. But they will still be called humanoid robots.
Totally agree. Wheels are cheaper, more durable and more effective than legs.
Human would have wheels if there was an evolution pathway to wheels.
- I use self-hosted gatus to monitor my certs and other services' status.
It can send alerts to multiple alerting providers.
- Stockfish has got rid of old handwritten evaluation now.
https://github.com/official-stockfish/Stockfish/pull/4674
Its evaluation now purely relies on NNUE neural network.
So it's an good exmaple of the better lesson. More compute evently won against handwritten evaluation. Stockfish developers thought old evaluation would help neural network so they kept the code for a few years, then it turned out that NNUE neural network didn't need any input of human chess knowledge.
- I don't use WARP's VPN mode.
I run WARP in socks proxy mode, and using ipt2socks for redirecting traffic to socks proxy port.
- Using Cloudflare WARP would be much faster.
And you can connect directly to ipv4 addr via WARP.
- Then if you offer your distilled model for commercial services, you would get sued by OpenAI in court.
- Distillation is great for researchers and hobbyists.
But nearly all frontier models have anti-distillation ToS, so distillation is out of question for western commercial companies like Apple.
- If you have worked or lived in China, you will know that Chinese open-source software industry is a totally shitshow.
The law in China offers little protection for open-source software. Lots of companies use open-source code in production without proper license, and there is no consequence.
Western internet influencers hype up Chinese open-source software industry for clicks while Chinese open-source developers are struggling.
These open-weight model series are planed as free-trial from the start, there is no commitment to open-source.
- Apple AI team keeps going against the bitter lesson and focusing on small on-device models.
Let's see how this would turn out in longterm.
- RDNA is a dead-end.
AMD went down the wrong path by focusing on traditional rendering instead of machine learning.
I think future AMD consumer GPUs would go back to GCN.
- These ppl are very loud online, but they don't make decisions for hyperscalers which are biggest spenders on AI chips.
AMD is doing just fine, Oracle just announced an AI cluster with up to 131,072 of AMD's new MI355X GPUs.
AMD needs to focus on bringing rack-scale mi400 as quickly as possible to market, rather than those hobbyists always find something to complain instead of spending money.
- I tried thinking with websearch on their website.
It has similar speed with o4-mini with search on chatgpt, and o4-mini gave me much better result.
- They seem to be very skeptical against Large models.
While everyone learned the bitter lesson, apple chose to focus on small on-device models even after the explosion of chatgpt.
- Oh, another LLM skepticism paper from Apple.
This paper from last year doesn't age well due to rapid proliferation of reasoning models.
- You can boot an OS on NVME drive from another bootloader on a USB drive.
I have been using CloverBootloader for years.
- Why no comparition to gpt-4o-transcribe？
If you don't compare to latest model on the market, how can you claim it's SOTA?
According to OpenAI, gpt-4o-transcribe has much better performance than whisper-large-v2.
https://openai.com/index/introducing-our-next-generation-aud...
After Transformer took over AI, FPGA for AI is totally dead now. Because Transformer is all about math matrix calculation, ASIC is the solution.
Modern Datacenter GPU is nearly AISC now.