Preferences

This is Max, one of the co-founders. We appreciate existing observability tools as they have saved us so much time in the past already. Excited to get your view on this! We've found many observability demands to be quite different when working on LLM applications. Mainly: Unpredictable input (users input free-form text that cannot be fully tested for), control flow highly dynamic when running on the textual output of a previous step and quality of output is not known at runtime (for the application it is just text). Many teams read manually through the LLM inputs and outputs to get a feeling for correctness or ask for user feedback. In addition, currently working on abstraction for model-based evals to make it simple to try which one works best for a use case and automatically run it on all production prompts/completions. One user described the difference to be that they use observability usually to know that nothing is going wrong whereas they use Langfuse many hours per day to understand how to best improve the application and navigate cost/latency/quality trade offs.

This item has no comments currently.

Keyboard Shortcuts

Story Lists

j
Next story
k
Previous story
Shift+j
Last story
Shift+k
First story
o Enter
Go to story URL
c
Go to comments
u
Go to author

Navigation

Shift+t
Go to top stories
Shift+n
Go to new stories
Shift+b
Go to best stories
Shift+a
Go to Ask HN
Shift+s
Go to Show HN

Miscellaneous

?
Show this modal