# A Model Gateway That Trains Its Router on Your Own Traces

> Experiential ingests your OpenTelemetry traces to build a router optimized against real agent traffic, then fine-tunes an open model on the same data.

Published: 2026-09-03
URL: https://daniliants.com/insights/a-model-gateway-that-trains-its-router-on-your-own-traces/
Tags: ai-gateway, open-source, observability, cost-optimization

---

## Summary

Experiential is an open source model gateway and router for agent workflows: one OpenAI-compatible API across hosted, BYOK, and local or custom LLMs, with per-user and per-agent access controls and spend budgets. It also turns production traffic, captured as OpenTelemetry traces, into a custom router or fine-tuned model optimized for quality, speed, and cost, available both as a local CLI and a hosted platform.

## Key Insight

- It goes beyond a simple proxy: `exp build <project>` ingests OTLP traces and simulates and optimizes a router against that real traffic, so routing decisions are trained on the agent's actual usage patterns rather than generic benchmarks.
- Closes the loop from observability to a bespoke model. `exp optimize model <project>` fine-tunes an open source model you own using Tinker, driven by the same collected traces.
- Same API surface locally and hosted: `pip install experiential && exp` runs a local OpenAI-compatible gateway, and the hosted endpoint is a drop-in equivalent (OpenAI-compatible plus Anthropic Messages API), so teams can start local and move to managed without changing client code.
- Cost governance is a first-class onboarding step, not bolted on. The setup wizard shows a default $50 command budget before issuing the one-time key.
- The repo ships copy-paste setup prompts (`SETUP.md`) explicitly meant to be handed to a coding agent so the agent configures its own gateway access, designed for agent-to-agent onboarding rather than just a human running the CLI.
- Telemetry is anonymous and aggregate via PostHog by default and explicitly excludes prompts, traces, actions, model names, and credentials, which is relevant for privacy review before adoption. Toggle it with `exp config telemetry disable/enable`.
- The router is embeddable, not just a standalone proxy: `exp.load_router("my-project")` in Python returns a project-scoped OpenAI client backed by its own private gateway instance.
- Test and dev conventions are standard modern Python tooling (`uv sync --extra dev`, `ruff format/check`, `ty check`, `pytest`), consistent with a codebase built to be extended.