TensorZero
TensorZero creates a feedback loop for optimizing LLM applications — turning production data into smarter, faster, and cheaper models.
It’s fully open source.
- Integrate our model gateway
- Send metrics or feedback
- Optimize prompts, models, and inference strategies
- Watch your LLMs improve over time
It provides a data & learning flywheel for LLMs by unifying:
- Inference: one API for all LLMs, with <1ms P99 overhead
- Observability: inference & feedback → your database
- Optimization: from prompts to fine-tuning and RL (& even 🍓? →)
- Experimentation: built-in A/B testing, routing, fallbacks
Demo — Data Driven NYC 2024
Watch LLMs get better at data extraction in real time with TensorZero.
Get started
Start building today. Check out our Github, Quick Start, or Tutorial.
Questions? Ask us on Slack or Discord.
Using TensorZero at work? Email us at hello@tensorzero.com to set up a Slack or Teams channel with your team (free).
Work with us. We’re hiring in NYC. We’d also welcome open-source contributions!