Hacker Neue

Preferences
dist-epoch parent
> This is a toy for rich people

GitHub copilot has a free tier.

Google gives you thousands of free LLM API calls per day.

There are other free providers too.

guappa
1st dose is free
palmfacehn
Agreed. It is worth noting how search has evolved over the years.
infecto
LLM APIs are pretty darn cheap for most of the developed worlds income levels.
guappa
Yeah, because they're bleeding money like crazy now.

You should consider how much it actually costs, not how much they charge.

How do people fail to consider this?

NitpickLawyer
No, there are 3rd party providers that run open-weights models and they are (most likely) not bleeding money. Their prices are kind of similar, and make sense in a napkin-math kind of way (we looked into this when ordering hardware).

You are correct that some providers might reduce prices for market capture, but the alternatives are still cheap, and some are close to being competitive in quality to the API providers.

Eggpants
Starts with “No” then follows that up with “most likely”.

So in other words you don’t know the real answer but posted anyways.

NitpickLawyer
That most likely is for the case where they made their investment calculations wrong and they won't be able to recoup their hw costs. So I think it's safe to say there may be the outlier 3rd party provider that may lose money in the long run.

But the majority of them are serving at ~ the same price, and that matches to the raw cost + some profit if you actually look into serving those models. And those prices are still cheap.

So yeah, I stand by what I wrote, "most likely" included.

My main answer was "no, ..." because the gp post was only considering the closed providers only (oai, anthropic, goog, etc). But youc an get open-weight models pretty cheap, and they are pretty close to SotA, depending on your needs.

Eggpants
Just wait for the enshitencation of LLM services.

It going to get wild when the tech bro investors demand ads be the included in responses.

It will be trivial for a version of AdWords where someone pays for response words be replaced. “Car” replaced by “Honda”, variable names like “index” by “this_index_variable_is_sponsered_by_coinbase” etc.

I’m trying to be funny with the last one but something like this will be coming sooner than later. Remember, google search used to be good and was ruined by bonus seeking executives.

bdangubic
how much does it cost?
infecto
>You should consider how much it actually costs, not how much they charge. How do people fail to consider this?

Sure, nobody can predict the long-term economics with certainty but companies like OpenAI already have compelling business fundamentals today. This isn’t some scooter startup praying for margins to appear; it’s a platform with real, scaled revenue and enterprise traction.

But yeah, tell me more about how my $200/mo plan is bankrupting them.

NoOn3
It's cheap now. But if you take into account all the training costs, then at such prices they cannot make a profit in any way. This is called dumping to capture the market.
infecto
No doubt the complete cost of training and to getting where we are today has been significant and I don’t know how the accounting will look years from now but you are just making up the rest based on feelings. We know operationally OpenAI is profitable on purely the runtime side, nobody knows how that will look when accounting for R&D but you have no qualification to say they cannot make a profit in any way.
guappa
Except they have to retrain constantly, so why would you not consider the cost of training?
infecto
In the medium to long term that R&D matters. In the short term it’s not as important of a metric. I absolutely agree from an underwriting prospective one would ideally be considering those costs but I also think it’s dishonest to simply say they are bleeding money, end of story.

They dont have to retrain constantly and that’s where opinions like yours fall short. I don’t believe anyone has a concrete vision on the economics in the medium to a long term. It’s biased ignorance to hold a strong position in the down or up case.

NoOn3
Yes, if you do not take into account the cost of training, I think it is very likely profitable. The cost of working models is not so high. This is just my opinion based on open models and I admit that I have not carried out accurate calculations.
diggan
> But if you take into account all the training costs

Not everyone has to paid that cost, as some companies are releasing weights for download and local use (like Llama) and then some other companies are going even further and releasing open source models+weights (like OLMo). If you're a provider hosting those, I don't think it makes sense to take the training cost into account when planning your own infrastructure.

Although I don't it makes much sense personally, seemingly it makes sense for other companies.

dist-epoch OP
There is no "capture" here, it's trivial to switch LLM/providers, they all use OpenAI API. It's literally a URL change.
jamessinghal
This is changing; OpenAI's newer API (Responses) is required to include reasoning tokens in the context while using the API, to get the reasoning summaries, and to use some of the OpenAI provided tools. Google's OpenAI compatibility supports Chat Completions, not Responses.

As the LLM developers continue to add unique features to their APIs, the shared API which is now OpenAI will only support the minimal common subset and many will probably deprecate the compatibility API. Devs will have to rely on SDKs to offer comptibility.

2 More Comments →
lelanthran
> There is no "capture" here, it's trivial to switch LLM/providers, they all use OpenAI API. It's literally a URL change.

So? That's true for search as well, and yet Google has been top-dog for decades in spite of having worse results and a poorer interface than almost all of the competition.

This item has no comments currently.