Preferences

I don't have experience with all of these systems. My observation is that whatever you do needs a bit of a systematic approach to evaluating the outcome. Without that you can use trial and error and replace magical solution 1 with magical solution 2 and hope for the best but it's probably not going to magically improve things. And blindly flailing around is effectively what a lot of companies are doing on this front. Most of my clients actually struggle with the basics. The short version of it is that you can probably make each of those systems do what you need probably but there's some skill to it. It's not about the tools you use but how you use them.

Opensearch is a fork of Elasticsearch and effectively the same thing for most things with exception of recently added features that a beginning user would typically not need. One of the exceptions is vector search which works slightly differently on both sides and mostly happened post fork.

There is no one size fits all solution for every search use case. Even realizing that there are multiple search use cases that you need to worry about and that they require different solutions is not something a lot of companies are even aware off. We can talk about text search, suggested queries, recommended results, did you mean, similarity search, etc. And that's before you throw in things like faceting/aggregations, various ways of putting your fingers on the scale (ever popular in ecommerce systems), premium/promoted content, etc.

Some wisdom with vector search is that it it is more effective when you combine it with traditional search. Doing vector search on lots of documents is just not cheap. So narrowing down results with a traditional search vastly cuts down on the amount of work you need to do. And it makes response times more reasonable too. Also, vector search with off the shelf models tends to perform quite poorly and models tend to have biases that might not match your needs/requirements. A good example from an ecommerce use case that somebody told me about is image search for clothes surfacing products that feature the same models rather than the same clothes. Because the off the shelf model was looking at the faces. It's not wrong. But probably not what was needed.

Lots of companies end up specializing models for their needs for this reason. And vector search is also not great for really short queries, like the type of queries that come out of a mobile UI. Users just don't tend to type coherently on mobile and there's only so much meaning you can get out of a 3 letter query. Vector search for search as you type is not a thing. And if you have a mobile UI that's most of the searches. Because why type long queries if the search gives you what you are looking for after just a few letters?

Most vector search products give you enough rope to hang yourself but don't fully solve all issues for you. Model selection and tuning tends to be all/most of the work and is typically left as an exercise to the user and is something that requires some skill. Once you have a model, there are plenty of things you can use. Each with their pros/cons. Setting up inference is the easy part of the job.

So, I wouldn't start with vector search until after you've done the legwork of being able to tell good results from bad results. Especially not if you are new to doing anything with search.

What is appropriate all depends on the use cases, the business value of search to your company, and the skills of the team that you have and what you are willing to spend on getting something better. But picking something because its "faster" because some silly benchmark says so is probably not a good reason. Is it faster because it skips essential work? Or because it does the same thing better? And is that the thing that you need? Or are there other things that you might need?


Fantastic insights again, thank you.

I've been developing a sort of educational platform for a few years and went deep on search stuff a couple years ago, before moving my focus elsewhere.

This was just prior to the LLM boom and I had concluded that it would probably be best to mostly avoid vector search, and instead use some sort of transformer model to extract summaries, keywords etc from each document, store them in a meta field, then just do normal, sparse bm25 on it all. Even for image search etc - just extract keywords rather then dense embeddings.

SPLADE was one promising example back then. https://github.com/naver/splade

I'm sure the field has progressed since then, but it sounds like it is still best to not invest in vector search.

The real lesson, it seems, is we need to know our needs, data, etc and act accordingly - most apparently do not do that.

This item has no comments currently.

Keyboard Shortcuts

Story Lists

j
Next story
k
Previous story
Shift+j
Last story
Shift+k
First story
o Enter
Go to story URL
c
Go to comments
u
Go to author

Navigation

Shift+t
Go to top stories
Shift+n
Go to new stories
Shift+b
Go to best stories
Shift+a
Go to Ask HN
Shift+s
Go to Show HN

Miscellaneous

?
Show this modal