Unified gateway for 250+ LLM models with observability, routing, and guardrails
Portkey documents an AI gateway at docs.portkey.ai that unifies access to more than 250 models through a Portkey SDK or OpenAI-compatible base URL (`PORTKEY_GATEWAY_URL`) with provider routing headers. Official quickstarts show three-line Python or TypeScript integrations that start monitoring LLM requests for resilience, security, and performance. Portkey states the open-source gateway is free to self-host while the managed service includes a free tier of 10k requests per month, edge-hosted workers adding roughly 20–40ms latency versus direct API calls, ISO 27001 and SOC 2 certifications, and optional configurations that skip storing request/response bodies.
Use cases
- Add gateway-level logging and failover without rewriting every provider SDK integration
- Route traffic across vendors using Portkey provider identifiers in headers
- Meet compliance reviews with documented ISO/SOC certifications and optional no-body storage
- Prototype quickly on the managed free tier before scaling request volume
- Keep OpenAI SDK code while inserting Portkey gateway headers via `createHeaders` helpers
Key features
- Portkey SDK and OpenAI-client gateway patterns documented with `x-portkey-api-key` and provider headers
- Unified interface spanning 250+ models per product overview
- Request monitoring, caching, and routing optimizations described in FAQs
- Enterprise options for managed private-cloud hosting and body-less logging modes
- SSO support with custom OIDC providers noted in documentation
Who Is It For?
- Platform engineers standardizing LLM access with observability baked in
- Teams running multi-model production apps that need routing and guardrails
- Startups seeking a quick gateway layer before building custom proxy infrastructure
Frequently Asked Questions
- Does Portkey replace provider API keys?
- Docs show both Portkey-native SDK usage and OpenAI-client patterns where upstream provider keys still authenticate the model vendor.
- What latency overhead should we expect?
- Portkey FAQs cite roughly 20–40ms additional latency from edge workers versus direct API calls.
- Is there a self-hosted option?
- Yes—the gateway is open source; managed hosting adds billing tiers starting with 10k free requests/month.
Related
Related
3 Indexed items
Helicone
Helicone documents an AI Gateway at ai-gateway.helicone.ai that lets teams call 100+ models from OpenAI, Anthropic, Google, Groq, and other vendors through an OpenAI-compatible base URL while logging every request to the Helicone dashboard. Official quickstart guides show signing up at helicone.ai, creating API keys in the US control plane, and pointing standard OpenAI SDK clients at the gateway with automatic observability. Helicone states credits carry 0% markup versus provider list prices, support automatic fallbacks when a provider is down, and allow bringing your own provider keys instead of using Helicone-managed credentials.
OpenRouter
OpenRouter is a model gateway that exposes many third-party AI models through one OpenAI-compatible API. Teams can compare providers, set routing preferences, and switch models without rewriting core client logic for each vendor SDK. The service publishes per-model pricing and supports pay-as-you-go usage.
LiteLLM
LiteLLM is an open-source Python library and proxy stack documented at docs.litellm.ai that exposes a single `completion()` interface across providers such as OpenAI, Anthropic, Vertex AI, Bedrock, and Ollama using OpenAI-compatible request and response shapes. The project documents a Router with retry, fallback, and load-balancing across deployments, optional observability callbacks (Langfuse, MLflow, Helicone, and others listed in observability guides), and a self-hosted LiteLLM Proxy (LLM Gateway) with virtual keys, spend tracking, guardrails, and an admin UI. Recent documentation also describes an MCP Gateway that centralizes MCP tool access with per-key, per-team, and per-organization permissions.