Data Model

The TensorZero Gateway stores inference and feedback data in ClickHouse. This data can be used for observability, experimentation, and optimization.

ChatInference

The ChatInference table stores information about inference requests for Chat Functions made to the TensorZero Gateway.

A ChatInference row can be associated with one or more ModelInference rows, depending on the variant’s type. For chat_completion, there will be a one-to-one relationship between rows in the two tables.

Tip

We anticipate adding more variant types in the future that might use multiple model inferences under the hood to generate a response (e.g. rejection sampling). See our Vision & Roadmap.

Column	Type	Notes
id	UUID	Must be a UUIDv7
function_name	String
variant_name	String
episode_id	UUID	Must be a UUIDv7
input	String (JSON)	input field in the /inference request body
output	String (JSON)	Array of content blocks
tool_params	String (JSON)	Object with any tool parameters (e.g. tool_choice, available_tools) used for the inference
inference_params	String (JSON)	Object with any inference parameters per variant type (e.g. {"chat_completion": {"temperature": 0.5}})
processing_time_ms	UInt32
timestamp	DateTime	Materialized from id (using UUIDv7ToDateTime function)

JsonInference

The JsonInference table stores information about inference requests for JSON Functions made to the TensorZero Gateway.

A JsonInference row can be associated with one or more ModelInference rows, depending on the variant’s type. For chat_completion, there will be a one-to-one relationship between rows in the two tables.

Tip

We anticipate adding more variant types in the future that might use multiple model inferences under the hood to generate a response (e.g. rejection sampling). See our Vision & Roadmap.

Column	Type	Notes
id	UUID	Must be a UUIDv7
function_name	String
variant_name	String
episode_id	UUID	Must be a UUIDv7
input	String (JSON)	input field in the /inference request body
output	String (JSON)	Object with parsed and raw fields
output_schema	String (JSON)	Schema that the output must conform to
inference_params	String (JSON)	Object with any inference parameters per variant type (e.g. {"chat_completion": {"temperature": 0.5}})
processing_time_ms	UInt32
timestamp	DateTime	Materialized from id (using UUIDv7ToDateTime function)

ModelInference

The ModelInference table stores information about each inference request to a model provider. This is the inference request you’d make if you had called the model provider directly.

Column	Type	Notes
id	UUID	Must be a UUIDv7
inference_id	UUID	Must be a UUIDv7
raw_request	String	Raw request as sent to the model provider (varies)
raw_response	String	Raw response from the model provider (varies)
model_name	String	Name of the model used for the inference
model_provider_name	String	Name of the model provider used for the inference
input_tokens	UInt32
output_tokens	UInt32
response_time_ms	UInt32
ttft_ms	Nullable(UInt32)	Only available in streaming inferences
timestamp	DateTime	Materialized from id (using UUIDv7ToDateTime function)

BooleanMetricFeedback

The BooleanMetricFeedback table stores feedback for metrics of type = "boolean".

Column	Type	Notes
id	UUID	Must be a UUIDv7
target_id	UUID	Must be a UUIDv7 that is either inference_id or episode_id depending on level in metric config
metric_name	String
value	Bool
timestamp	DateTime	Materialized from id (using UUIDv7ToDateTime function)

FloatMetricFeedback

The FloatMetricFeedback table stores feedback for metrics of type = "float".

Column	Type	Notes
id	UUID	Must be a UUIDv7
target_id	UUID	Must be a UUIDv7 that is either inference_id or episode_id depending on level in metric config
metric_name	String
value	Float32
timestamp	DateTime	Materialized from id (using UUIDv7ToDateTime function)

CommentFeedback

The CommentFeedback table stores feedback provided with metric_name of "comment". Comments are free-form text feedbacks.

Column	Type	Notes
id	UUID	Must be a UUIDv7
target_id	UUID	Must be a UUIDv7 that is either inference_id or episode_id depending on level in metric config
target_type	Enum(inference, episode)
value	String
timestamp	DateTime	Materialized from id (using UUIDv7ToDateTime function)

Tip

We’ll soon also support a new metric type: demonstrations. Read more in Vision & Roadmap.