Overview

TensorZero Recipes are a set of pre-built workflows for optimizing your LLM applications. You can also create your own recipes to customize the workflow to your needs.

The TensorZero Gateway collects structured inference data and the downstream feedback associated with it. This dataset sets the perfect foundation for building and optimizing LLM applications. As this dataset builds up, you can use these recipes to generate powerful variants for your functions. For example, you can use this dataset to curate data to fine-tune a custom LLM, or run an automated prompt engineering workflow.

In other words, TensorZero Recipes optimize TensorZero functions by generating new variants from historical inference and feedback data.

Recipes

Supervised Fine-tuning

A fine-tuning recipe curates a dataset from your historical inferences and fine-tunes an LLM on it. You can use the feedback associated with those inferences to select the right subset of data. A simple example is to use only inferences that led to good outcomes according to a metric you defined.

We present two sample fine-tuning recipes:

• Fine-tuning with Fireworks AI
• Fine-tuning with OpenAI

See complete examples using the recipes below.

Dynamic In-Context Learning

Dynamic In-Context Learning (DICL) is a technique that leverages historical examples to enhance LLM performance at inference time. It involves selecting relevant examples from a database of past interactions and including them in the prompt, allowing the model to learn from similar contexts on-the-fly. This approach can significantly improve the model’s ability to handle specific tasks or domains without the need for fine-tuning.

We provide a sample recipe for DICL with OpenAI. The recipe supports selecting examples based on boolean metrics, float metrics, and demonstrations.

• Dynamic In-Context Learning with OpenAI

Tip

Many more recipes are on the way. This will be our primary engineering focus in the coming months. We also plan to publish a dashboard that’ll further streamline some of these recipes (e.g. one-click fine-tuning).

Read more about our Vision & Roadmap.

Custom Recipes

You can also create your own recipes.

Put simply, a recipe takes inference and feedback data stored that the TensorZero Gateway stored in your ClickHouse database, and generates a new set of variants for your functions. You should should be able to use virtually any LLM engineering workflow with TensorZero, ranging from automated prompt engineering to advanced RLHF workflows. See an example of a custom recipe using DSPy below.

Examples

We are working on a series of complete runnable examples illustrating TensorZero’s data & learning flywheel.

• Writing Haikus to Satisfy a Judge with Hidden Preferences — This example fine-tunes GPT-4o Mini to generate haikus tailored to a specific taste. You’ll see TensorZero’s “data flywheel in a box” in action: better variants leads to better data, and better data leads to better variants. You’ll see progress by fine-tuning the LLM multiple times.
• Improving Data Extraction (NER) by Fine-Tuning a Llama 3 Model — This example shows that an optimized Llama 3.1 8B model can be trained to outperform GPT-4o on a Named Entity Recognition (NER) task using a small amount of training data, and served by Fireworks at a fraction of the cost and latency.
• Improving Data Extraction (NER) with Dynamic In-Context Learning — This example demonstrates how Dynamic In-Context Learning (DICL) can enhance Named Entity Recognition (NER) performance by leveraging relevant historical examples to improve data extraction accuracy and consistency without having to fine-tune a model.
• Improving LLM Chess Ability with Best-of-N Sampling — This example showcases how best-of-N sampling can significantly enhance an LLM’s chess-playing abilities by selecting the most promising moves from multiple generated options.
• Improving Math Reasoning with a Custom Recipe for Automated Prompt Engineering (DSPy) — TensorZero provides a number of pre-built optimization recipes covering common LLM engineering workflows. But you can also easily create your own recipes and workflows! This example shows how to optimize a TensorZero function using an arbitrary tool — here, DSPy.