2
points
stanleydrew
Joined 8,502 karma
blog: https://benton.io
gmail: andrewmbenton
Co-founder and CEO at Riza, a company working on untrusted code isolation with WASM: https://riza.io / hello@riza.io
Previously COO at ngrok: https://ngrok.com.
I was the Founder and CEO of an MVNO/VoIP hybrid company called Charge (formerly known as Bolt): https://charge.co.
I worked at Twilio: https://www.twilio.com.
I wrote the original ruby library that wraps Twilio's HTTP API: https://github.com/twilio/twilio-ruby
Other projects include receive-only telephone numbers at https://oneway.tel and free HPKP report bin hosting at https://hpkp.org (now defunct).
[ my public key: https://keybase.io/andrewmbenton; my proof: https://keybase.io/andrewmbenton/sigs/m4O8mq4uxoPSm2XlB1RUT1XkH5v8KX7UR448y8T74mo ]
- > idk why we need MCP servers when LLMs can just connect to the existing API endpoint
Because the LLM can't "just connect" to an existing API endpoint. It can produce input parameters for an API call, but you still need to implement the calling code. Implementing calling code for every API you want to offer the LLM is at minimum very annoying and often error-prone.
MCP provides a consistent calling implementation that only needs to be written once.
- Technically it's not really much different from just giving the LLM an OpenAPI spec.
The actual thing that's different is that an OpenAPI spec is meant to be an exhaustive list of every endpoint and every parameter you could ever use. Whereas an MCP server, as a proxy to an API, tends to offer a curated set of tools and might even compose multiple API calls into a single tool.
- > I have a silly theory that I only half joke about that docker/containers wouldn't've ever taken off as fast as it did if it didn't solve the horrible python dependency hell so well.
I don't think this is a silly theory at all. The only possibly silly part is that containers specifically helped solve this problem just for python. Lots of other software systems built with other languages have "dependency hell."
- > Is this just junior devs who never learned it
Seems more like it's fallen out of favor with senior devs who have moved to Go/Rust.
- 3 points
- 2 points
- What risk do you foresee arising out of perverse incentives in this case?
- 2 points
- Also means you're not having to do a bunch of isolation work to make the server-side execution environment safe.
- 3 points
- One of the design principles of sqlc is that SQL queries should be static in application code so that you know exactly what SQL is running on your database. It turns out you can get pretty far operating under this constraint, although there are some annoyances.
- 9 points
- 4 points
- Riza, Inc. (https://riza.io) | SWEs and DevRel Engineers | Full-time or part-time | San Francisco
We use WASM to provide isolated runtimes for executing untrusted code, mostly generated by LLMs. Our customers do things like extract data from log lines at run time by asking claude-3-5-sonnet to generate a parsing function on-the-fly and then sending it to us for execution.
Things we need help with:
* Our janky account management dashboard (Postgres / Go / React / TypeScript)
* Our hosted and self-hosted runtime service (Rust, WASM)
* Integrations and demos with adjacent frameworks and tools (Python / JavaScript / TypeScript)
* New products
We have seed money, but the whole company is currently just me and Kyle working out of a converted warehouse on Alabama St. We’re second-time founders, so we know the risk we’re asking you to take and we’re prepared to compensate accordingly. Send an email to me at andrew at riza dot io or pop in our Discord (https://discord.gg/4P6PUeJFW5) and say hi.
- Why do we have to "get there?" Humans use calculators all the time, so why not have every LLM hooked up to a calculator or code interpreter as a tool to use in these exact situations?
- I would argue that most sota models do know that they don't know this, as evidenced by the fact that when you give them a code interpreter as a tool they choose to use it to write a script that counts the number of letters rather than try to come up with an answer on their own.
(A quick demo of this in the langchain docs, using claude-3-haiku: https://python.langchain.com/v0.2/docs/integrations/tools/ri...)
- Yes, we are doing this at Riza[0] (via WASM). I'd love to have folks try our downloadable CLI which wraps isolated Python/JS runtimes (also Ruby/PHP but LLMs don't seem to write those very well). Shoot me an email[1] or say hi in Discord[1].
[0]:https://riza.io [1]:mailto:andrew@riza.io [2]:https://discord.gg/4P6PUeJFW5
- Plug in a code interpreter as a tool and the model will write Python or JavaScript to solve this and get it right 100% of the time. (Full disclosure: I work on a product called Riza that you can use as a code interpreter tool for LLMs)
Presumably over time shared parking areas will get upgraded with charging infrastructure to keep attracting tenants.