Getting Started with Google AI Studio (Gemini API)

This guide shows how to set up a minimal deployment to use the TensorZero Gateway with Google AI Studio (Gemini API).

Setup

For this minimal setup, you’ll need just two files in your project directory:

Directoryconfig/
- tensorzero.toml
docker-compose.yml

For production deployments, see our Deployment Guide.

Configuration

Create a minimal configuration file that defines a model and a simple chat function:

[models.gemini_1_5_flash_8b]
routing = ["google_ai_studio_gemini"]

[models.gemini_1_5_flash_8b.providers.google_ai_studio_gemini]
type = "google_ai_studio_gemini"
model_name = "gemini-1.5-flash-8b"

[functions.my_function_name]
type = "chat"

[functions.my_function_name.variants.my_variant_name]
type = "chat_completion"
model = "gemini_1_5_flash_8b"

# Disable observability to keep this example minimal (not recommended in production)
[gateway]
disable_observability = true

See the list of models available on Google AI Studio (Gemini API).

Credentials

You must set the GOOGLE_AI_STUDIO_API_KEY environment variable before running the gateway.

Deployment (Docker Compose)

Create a minimal Docker Compose configuration:

# This is a simplified example for learning purposes. Do not use this in production.
# For production-ready deployments, see: https://www.tensorzero.com/docs/gateway/deployment

services:
  gateway:
    image: tensorzero/gateway
    volumes:
      - ./config:/app/config:ro
    environment:
      - GOOGLE_AI_STUDIO_API_KEY=${GOOGLE_AI_STUDIO_API_KEY:?Environment variable GOOGLE_AI_STUDIO_API_KEY must be set.}
    ports:
      - "3000:3000"

You can start the gateway with docker compose up.

Inference

Make an inference request to the gateway:

curl -X POST http://localhost:3000/inference \
  -H "Content-Type: application/json" \
  -d '{
    "function_name": "my_function_name",
    "input": {
      "messages": [
        {
          "role": "user",
          "content": "What is the capital of Japan?"
        }
      ]
    }
  }'