67
points
roma_glushko
Joined 145 karma
[ my public key: https://keybase.io/roman_hlushko; my proof: https://keybase.io/roman_hlushko/sigs/aR2rnIlbGU8xh46dI5b4lH0K4ffRftUjDULjmiwJnVM ]
- A new OSS framework uses multi-objective Bayesian Optimization to efficiently search for Pareto-optimal RAG workflows, balancing cost, accuracy, and latency across configurations that would be impossible to test manually.
- 341 points
- The article covers:
- High-level architecture of the platform & the main data model - Handling communication with customers across 7 channels in a uniformed way and figure out to which ongoing conversation it belongs - How to receive emails via AWS SES - How to sync emails from GMail - How to receive SMS via Twilio - Embedded Chat Capabilities via Phoenix Channels & Presence - Usage of Slack/Mattermost as reply channels - Webhook and Lambda Function functionality to extend the existing functionality - Advanced browser session tracking and reply - Reporting, analytics, billing strategy
Papercups is an excellent example of how to build a SaaS platform in a radically simple way.
- The article covers: - The Signal Processing Pipeline Architecture - OTel Receivers. Prometheus-style Scrapers - OTel Processors. The Memory Limiter & Batch Processor. Multi-tenant Signal Processing - OTel Exporters. The Exporting Pipeline & Queues. The implementation of persistent queues - How observability is done in the OTel Collector itself. Logging, metrics, and traces - OTel Extensions Design. Authentication & ZPages -Custom Collectors & OTel Collector Builder - Feature Gates Design & The Feature Release & Deprecation Process
- 1 point
- An article for those who want to go a little bit beyond the basic usage of OTEL SDK and understand how it works under the hood.
The post quickly touches on these topics:
- History and the idea of OpenTelemetry - Distributed traces & spans. How span collection happens on the service side - Baggage & trace ctx propagation - Metrics collection. Views & aggregations. Metrics readers - OTEL Logging integration - Semantic conventions and why that is important
Let me know what do you think
- Glide strives to help you to solve common problems that occur during development and running GenAI apps by moving them out of your specific applications on the level of your infrastructure. All you need to do to start leveraging that is to talk to your models via Glide
As a part of this initial scope, we had to set up a bunch of common things to make it roll. As for the core functionality, we have brought up:
- The routing functionality with four types of routing strategies (including a tricky one like the least latency routing) - The first-class adaptive resiliency & fallbacking across all routing strategies - Unified Chat API that supports popular model providers like OpenAI, Azure OpenAI (on-prem models), Cohere, OctoML, Anthropic - The ability to have model-specific prompts - Installation via Docker & Homebrew
- 3 points
- Yeah, papermill is a go-to tool for this usecase
- Despite having many separate libraries in PyPi that try to implement some of the listed patterns, few would be considered production-ready and practically useful. Most are missing important features, configurations and limits that make them inconvenient or dangerous to use in production under the scale.
This is the problem that Hyx is trying to solve.
Hyx is a slim resiliency toolkit for asyncio-based Python microservices. The idea is to implement all industry-proven stability patterns in a safe production-ready way by incorporating experience shared by industry leaders like Google, Amazon, Neflix, Microsoft. So you don't think twice what library to install to improve your resiliency situation.
It's pretty much like Hystrix, resilience4j or Polly, but for Python.
At this point, the project has been inited with the basic implementations and docs for the following components:
- retries (with several backoff and jitter strategies)
- consecutive circuit breaker
- timeouts
- bulkhead (concurrency limiter)
- Token Bucket-based Rate limiter
- fallback
On the way to make it production ready, I plan to add support for metrics dispatching (via OpenTelemetry, Prometheus, StatsD), implement more variants of circuit breakers, rate limiters, etc, make existing components more ready-to-use and tailored to the common use cases, provide examples and integrations with popular frameworks (like FastAPI/Starlette, Flask, etc)
If the idea sounds useful, feel free to give it star. If you would like some functionality to be supported, drop me a message here or in the Github Discussions. That will help me to prioritise the work.
Links:
- Github: https://github.com/roma-glushko/hyx
- 2 points
- Inclusive guide for the non-UI engineers who would like to understand hype around hooks in 5 mins
- Ready to play? Go and check out my demo of the rock, paper, scissors game powered by Tensorflow.js.
Github: https://github.com/datarobot/syftr
Paper: https://arxiv.org/abs/2505.20266