Formerly Google Play App bundling and delivery Formerly at Thread (thread.com).
Opinions here do not reflect the opinion of my employer.
https://danpalmer.me/
- And what is a "user"?
- I did. I found it to have quite a few problems with bugs, docs, and web page lifecycle.
I switched to Hotwire/Stimulus and found it to be a significantly better implementation of the same core concepts. I'd highly recommend checking them out.
- How do you define a user, and how do you define online?
If the forum considers unique cookies to be a user and creates a new cookie for any new cookie-less request, and if it considers a user to be online for 1 hour after their last request, then actually this may be one scraper making ~6 requests per second. That may be a pain in its own way, but it's far from 23k online bots.
- To be clear I'm only suggesting that hardware is a factor here, it's far from the only reason. The parent commenter corrected their comment that it was actually Groq not Grok that they were thinking of, and I believe they are correct about that as Groq is doing something similar to TPUs to accelerate inference.
- The link is just to the book, the details are scattered throughout. That said the page on GPUs specifically speaks to some of the hardware differences and how TPUs are more efficient for inference, and some of the differences that would lead to lower latency.
https://jax-ml.github.io/scaling-book/gpus/#gpus-vs-tpus-at-...
Re: Groq, that's a good point, I had forgotten about them. You're right they too are doing a TPU-style systolic array processor for lower latency.
- I thought it was generally accepted that inference was faster on TPUs. This was one of my takeaways from the LLM scaling book: https://jax-ml.github.io/scaling-book/ – TPUs just do less work, and data needs to move around less for the same amount of processing compared to GPUs. This would lead to lower latency as far as I understand it.
The citation link you provided takes me to a sales form, not an FAQ, so I can't see any further detail there.
> Both Cerebras and Grok have custom AI-processing hardware (not CPUs).
I'm aware of Cerebras' custom hardware. I agree with the other commenter here that I haven't heard of Grok having any. My point about knowledge grounding was simply that Grok may be achieving its latency with guardrail/knowledge/safety trade-offs instead of custom hardware.
- My understanding is that TPUs do not use memory in the same way. GPUs need to do significantly more store/fetch operations from HBM, where TPUs pipeline data through systolic arrays far more. From what I've heard this generally improves latency and also reduces the overhead of supporting large context windows.
- Hardware is a factor here. GPUs are necessarily higher latency than TPUs for equivalent compute on equivalent data. There are lots of other factors here, but latency specifically favours TPUs.
The only non-TPU fast models I'm aware of are things running on Cerebras can be much faster because of their CPUs, and Grok has a super fast mode, but they have a cheat code of ignoring guardrails and making up their own world knowledge.
- Thank you, I think it might have been collision avoidance and/or attitude control that I was thinking of then, rather than actually burning up. I remember reading about this in relation to the ISS which needs frequent adjustments, although is a bit lower than 500km.
- I know they charge for Artifact storage, but outside of uploaded artifacts I don't think that the logs and results of builds are billed separately?
Additionally, I thought that caching came out of a separate limit, and was not billed like artifact storage?
- > For them to do essentially nothing.
Orchestration, logging, caching, result storage.
It's not nothing. Whether it's worth it to you is a value judgement, and having run a bunch of different CI systems I'd say this is still at least competitive.
- My understanding was that anything at ~500km needed readjustments every few months in order to not come down. Much less than 2-3 years.
I'd be interested to know what the average lifespan or failure rate of Starlink has been. That's good that some are still up there 6+ years later, but I know many aren't. I'm not sure how many of those ran out of fuel, had hardware failures, or were simply obsolete, but an AFR would be interesting to see.
- Starlink is however operating at ~500km where radiation is less of a concern, but where the lifetime of a satellite is only 2-3 years.
The unit economics of orbital GPUs suggest that we'll need to run them for much longer than that. This is actually one of the few good points of orbital data centers, normally older hardware is cycled out because it's not economic to run anymore due to power efficiency improvements, but if your power is "free" and you've already got sufficient solar power onboard for the compute, you can just keep running old compute as long as you can keep the satellite up there.
- There's more than that, it's possible to get permanent hardware damage from radiation at smaller (modern standard) process sizes.
- Legislation. If a country requires age verification, identity verification, moderation, etc, it's easy enough to either block that traffic or enforce the local laws. However users can easily circumvent this with a VPN. For some countries, this traffic is still in scope, and so the only real way to prevent it is to block or impose the restrictions on all VPN users.
Could also be spam/abuse prevention. Credential stuffing often goes through VPNs, signup over VPN is a strong signal for future abuse or issues in various ways.
- It's worth being very clear about the reasons for it. You say you're not trying to build a media business, and want to build a "public notebook", but you imply you're looking for career/networking opportunities. You're also not clear if the "learning" is for you as the author, or for the reader.
If you want to have your content discovered online, I'd say you might be in for some trouble, although I don't think AI is the cause, only an accelerator on that. Blogs for readers learning are probably in decline, you're unlikely to get any outreach based on your posts for networking.
However if, like me, the writing process is the point – you're trying to clarify your thoughts, learn something new yourself, or have a document you can share with colleagues when they ask you to explain your opinions, I think blogging is valuable. While you won't get direct outreach, you can share it on your CV or send it to recruiters and you might get noticed when applying for jobs.
- > Most people using excel and word would be just as functional using office '98.
This is just not at all true of the professional world. Home and student use, sure, but in business the office suite is deeply tied into workflows, business processes, approvals, review flows, version control, data analysis, data warehouses, and so much more.
There are companies that for all intents and purposes run on Excel. This goes far beyond spreadsheets, that's just the interface, it's the live data backing onto other services, it's the plugin systems, etc. My previous company ran significant processes on Google Sheets with a lot of automation built around it.
And then there's Sharepoint and all of that, all the sharing and access control is baked through the stack and available in all the frontends, whether that's on desktop, mobile, web, etc.
None of this was around in Office '98. There were some very early reaches into these sorts of things, but they would be unrecognisable now. We've progressed nearly 30 years after all.
- > the not-so-hidden assumption is that for any given domain, the efficiency asymptote peaks well above the alternative
This is an assumption for the best-case scenario, but I think you could also just take the marginal case. Steady progress builds until you get past the state of the art system, and then the switch becomes easy to justify.
- This is a failure of business model and logistics, not a failure of the robotics.
> Fenyo added that Kroger’s decision to locate the Ocado centers outside of cities turned out to be a key flaw.
They over-spent on automating low-volume FCs. You could draw comparisons to Amdahl's law, they optimised the bit that wasn't the issue, the real issue was delivery distances and times.
Ocado has had good success with the robotics approach in the UK, because the UK is very high density compared to a lot of the US. Plus Ocado put a lot of work into creating good delivery routes, whereas it sounds like that wasn't a component of the automation stack that Kroger bought.
Couple that with 15 minute session times, and that could just be one entity scraping the forum at 30 requests per second. One scraper going moderately fast sounds far less bad than 29000 bots.
It still sounds excessive for a niche site, but I'd guess this is sporadic, or that the forum software has a page structure that traps scrapers accidentally, quite easy to do.