Respan, a YC W24 company, released its open-source Rust AI gateway built around plugins, first presented in a Hacker News launch thread that reached 50 points and 20 comments.
The project positions itself as a lightweight proxy layer for routing requests to multiple LLM providers with custom logic added through plugins.
Project: Respan | Language: Rust | License: Open source | Focus: Plugin-based AI gateway | Origin: YC W24
What It Is and How It Works
Respan acts as an AI gateway that sits between applications and LLM endpoints. Its core design uses a plugin system so developers can insert routing, logging, rate limiting, or provider-specific transformations without modifying the gateway binary.
Because it is written in Rust, the gateway targets low memory overhead and fast request handling compared with higher-level language alternatives.
Community Reaction on Hacker News
The launch thread collected 50 points and 20 comments within the first day. Commenters focused on three recurring points:
- Interest in Rust for production LLM proxies due to safety and speed
- Questions about plugin API stability and versioning
- Requests for built-in support for common providers such as OpenAI, Anthropic, and local models
Early testers noted the plugin model allows teams to keep sensitive routing logic inside their own compiled modules rather than relying on configuration files alone.
How to Try It
The project is distributed as open-source code. Developers can clone the repository and build the binary with standard Rust tooling. Plugins are loaded at startup from a designated directory, following the documented interface.
No hosted service is mentioned in the launch materials; usage requires self-hosting.
Pros and Cons
-
Pros
- Rust implementation keeps memory footprint small under concurrent load
- Plugin architecture supports custom logic without forking the core
- Open-source license allows inspection and modification
-
Cons
- Requires Rust compilation knowledge for custom plugins
- Limited provider integrations at launch compared with mature gateways
- No managed cloud option announced
Alternatives and Comparisons
Several existing tools handle LLM routing and gateway functions.
| Feature | Respan | LiteLLM | Kong AI Gateway |
|---|---|---|---|
| Core language | Rust | Python | Lua / Go |
| Plugin model | Compiled plugins | Python hooks | Lua plugins |
| Memory overhead | Low | Higher | Medium |
| Self-hosted | Yes | Yes | Yes |
| YC affiliation | W24 | None | None |
Respan differentiates itself through the compiled-plugin approach and Rust performance claims, while LiteLLM offers broader provider coverage out of the box.
Who Should Use This
Teams already running Rust services or needing strict control over request handling logic will find the plugin model useful. Organizations that prefer managed gateways or require extensive pre-built provider integrations should evaluate LiteLLM or commercial options first.
Bottom line: Respan offers a Rust-native, plugin-driven gateway for teams that want to embed custom LLM routing logic directly into a compiled binary.
The project’s early traction on Hacker News suggests demand for lightweight, language-native alternatives in the AI gateway space.
Top comments (0)