Requesty: AI Gateway & LLM Router for 600+ Models

My notes

Summary

Requesty is an AI gateway that sits between an app and 600+ LLM providers behind one OpenAI-compatible endpoint. It adds intelligent routing, automatic failover, semantic caching, geo-based data residency, spend and policy controls, and PII scrubbing without requiring changes to existing SDK code. Pricing is a flat 5% markup on the underlying model cost.

Key Insight

• Drop-in integration: switching providers or models is a one-line change, just point the OpenAI SDK’s base_url at router.requesty.ai/v1 and keep using client.chat.completions.create() as normal; model choice becomes a string like "anthropic/claude-sonnet-4-20250514".
• Failover speed matters more than people assume: they claim sub-14ms provider switchover on outage detection, with a real example of 3 failovers in a month and zero user-facing downtime. This is the kind of SLA number worth demanding from any router under evaluation.
• Semantic caching has real ROI: one customer dashboard shows 52,800 cache hits saving $462 per month and pulling total spend down from $1,422. Caching is not just for identical repeated prompts, it matches semantically similar ones too.
• Policy cascades, not flat rules: spending limits, model allowlists, and rate limits cascade from organization to team to individual API key, which maps well onto how agencies and consultancies with multiple clients and multiple budgets actually need to segment AI spend.
• Agent-level routing is a distinct feature from user-level routing: per-agent preferred models, fallback chains, and cost caps are relevant for anyone running multi-agent systems where different agents have different quality and cost tradeoffs, for example a cheap classifier agent versus an expensive reasoning agent.
• Governance features baked in, not bolted on: automatic PII detection and redaction (emails, phone numbers, SSNs, credit cards) happens before the prompt reaches the model, useful when compliance is the blocker to adopting LLMs in a workflow.
• Pricing model is simple but has a real cost: a flat 5% markup on model costs, with bring-your-own-key or a unified key, is the tradeoff for not building routing, observability, and failover in-house.