- I’m finishing up a language identification model that runs on cpu, 70k texts/s single thread, 13mb model artifact and 148 supported languages (though only ~100 have good accuracy).
This is a model trained as static embeddings from the gemma 3 token embeddings.
- > Berulis said he and his colleagues grew even more alarmed when they noticed nearly two dozen login attempts from a Russian Internet address (83.149.30,186) that presented valid login credentials for a DOGE employee account
> “Whoever was attempting to log in was using one of the newly created accounts that were used in the other DOGE related activities and it appeared they had the correct username and password due to the authentication flow only stopping them due to our no-out-of-country logins policy activating,” Berulis wrote. “There were more than 20 such attempts, and what is particularly concerning is that many of these login attempts occurred within 15 minutes of the accounts being created by DOGE engineers.”
https://krebsonsecurity.com/2025/04/whistleblower-doge-sipho...
I’m surprised this didn’t make bigger news.
- Heavens to Betsy, I don’t know if you can hear me, But try supporting these things if you actually want them to be successful. About the 3rd day into trying to roll your own LMI container in sagemaker because they haven’t updated the vLLM version in 6 months and you can’t run a regular sagemaker endpoint because of a ridiculous 60s timeout that was determined to be adequate 8 years ago. I can only imagine the hell that awaits the developer that decides to try their custom silicon.
- That might not be relevant to OPs use case. A lot of nurses get tied up doing things like reviewing claims denials. There’s good use cases on the administrative side of healthcare that currently require nurse involvement.
- I love using encoder models, and they are generally a better technology for this kind of application. But the price of GPU instances is too damn high.
I won’t lie that I’ve been unreasonably annoyed that I have to use a lot more compute than I need, for no other reason than an LLM API exists and it’s good enough in a relatively small throughput application.
- One of the issues with using LLMs in content generation is that instruction tuning causes mode collapse. For example, if you ask an LLM to generate a random number between 1 and 10, it might pick something like 7 80% of the time. Base models do not exhibit the same behavior.
“Creative Output” has an entirely different meaning when you start to think about them in the way they actually work.
- SPLADE-easy: https://github.com/dleemiller/splade-easy
I wanted a simple retrieval index to use splade sparse vectors. This just encodes and serializes documents into flatbuffers and appends them into shards. Retrieval is just parallel flat scan, optionally with reranking.
The idea is just a simple, portable index for smaller data sizes. I’m targeting high quality hybrid retrieval, for local search, RAG or deep research scenarios.
SPLADE is a really nice “in-between” for semantic and lexical search. There’s bigger and better indexes out there like Faiss or Anserini, but I just kinda wanted something basic.
I was testing it on 120k docs in a simple cli the other day and it’s still as good as any web search experience (in terms of latency) — so I think it’ll be useful.
We’re still trying to clean up the API and do a thorough once over, so I’m not sure I’d recommend trying it yet. Hopefully soon.
- I think that happened when gpt5 was released and pierced OpenAIs veil. While not a bad model, we found out exactly what Mr. Altman’s words are worth.
- I haven’t used RCNN, but trained a custom YOLOv5 model maybe 3-4 years ago and was very happy with the results.
I think people have continued to work on it. There’s no single lab or developer, it mostly appears that the metrics for comparison are usually focused on the speed/MAP plane.
One nice thing is that even with modest hardware, it’s low enough latency to process video in real time.
- FWIW this has happens in consulting too, not just product companies. Just swap “product” for “delivery”.
- I think a less order biased, more straightforward way would be just to vectorize everything, perform clustering and then label the clusters with the LLM.
- > For the searches we use hybrid dense + sparse bm25, since dense doesn't work well for technical words.
One thing I’m always curious about is if you could simplify this and get good/better results using SPLADE. The v3 models look really good and seem to provide a good balance of semantic and lexical retrieval.
- Absolutely the first thing you should try is a prompt optimizer. The GEPA optimizer (implemented in DSPy) often outperforms GRPO training[1]. But I think people are usually building with frameworks that aren't machine learning frameworks.
- I go back and forth on this. A year ago, I was optimistic and I have had 1 case where RL fine tuning a model made sense. But while there are pockets of that, there is a clash with existing industry skills. I work with a lot of machine learning engineers and data scientists and here’s what I observe.
- many, if not most MLEs that got started after LLMs do not generally know anything about machine learning. For lack of clearer industry titles, they are really AI developers or AI devops
- machine learning as a trade is moving toward the same fate as data engineering and analytics. Big companies only want people using platform tools. Some ai products, even in cloud platforms like azure, don’t even give you the evaluation metrics that would be required to properly build ml solutions. Few people seem to have an issue with it.
- fine tuning, especially RL, is packed with nuance and details… lots to monitor, a lot of training signals that need interpretation and data refinement. It’s a much bigger gap than training simpler ML models, which people are also not doing/learning very often.
- The limited number of good use cases means people are not learning those skills from more senior engineers.
- companies have gotten stingy with sme-time and labeling
What confidence do companies have in supporting these solutions in the future? How long will you be around and who will take up the mantle after you leave?
AutoML never really panned out, so I’m less confident that platforming RL will go any better. The unfortunate reality is that companies are almost always willing to pay more for inferior products because it scales. Industry “skills” are mostly experience with proprietary platform products. Sure they might list “pytorch” as a required skill, but 99% of the time, there isn’t hardly anyone at the company that has spent any meaningful time with it. Worse, you can’t use it, because it would be too hard to support.
- > “What would America’s Founding Fathers think if they were alive today?”
> For Cross, it is pointless to speculate about the present-day views of men who could not have imagined cotton candy, let alone the machine that makes it.
Some things, like “taxation without representation” seem to be timeless. You can call it irony or perhaps in some cases, a spade is still just a spade.
- David Frum talked at length about self-abasement in MAGA public culture in his recent podcast for the Atlantic[1].
> I think it also becomes a real test of in-group loyalty to see who can outcompete in slavishness the other members of the circle, who are also competing to be slavish. That’s why you get these strange [phenomena] like Donald Trump’s physicians claiming that he’s the most physically vigorous president ever.
> Now, even when Donald Trump was younger, he was a big man, but he was never a great athlete. And now, as he approaches his 80th birthday, he’s obviously not physically fit.
> The fact is, you’re not just willing to tell a lie, but tell a lie that abases you, that makes you look foolish, that makes you look like you don’t care about yourself at all, that you only defer to the leader. That’s the real sign of loyalty. It’s flattery that is not meant to be believed but functions as a kind of system of in-group recognition.
To me, this is a perfect mirror to Chairman Mao (supposedly) swimming across the Yangtze River in his 70s at a pace faster than an Olympic champion of today.
There’s no meaning to any of it. It’s just propaganda and self-abasement for the purpose of loyalty competition to the leader. In fact, the more ludicrous, the better, because it means you’re willing to fully destroy any personal credibility you may have as a sacrifice to show loyalty.
[1] https://www.theatlantic.com/podcasts/archive/2025/10/the-dav...
- It’s a result of the lack of rigor in how it’s being used. Machine learning has been useful for years despite less than 100% accuracy, and the way you trust it is through measurement. Most people using or developing with AI today have punted on that because it’s hard or time consuming. Even people who hold titles of machine learning engineer seem to have forgotten.
We will eventually reach a point where people are teaching each other how to perform evaluation. And then we’ll probably realize that it was being avoided because it’s expense to even get to the point where you can take a measurement and perhaps you didn’t want to know the answer.
- This seems like a great place for a Cypress (Infineon) PSOC. A while back, I interfaced one to a linear CCD and it was a great experience. They also have USB HID support on chip.
Eventually we will have to somehow convince AI of new and better ways of doing things. It’ll be propaganda campaigns waged by humans to convince God to deploy new instructions to her children.