> For the complete documentation index, see [llms.txt](https://docs.wayfound.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.wayfound.ai/ai-gateways/connecting-litellm-gateway.md). # Connecting LiteLLM Gateway ## Supervising Agents Through Your LiteLLM Gateway If your LLM traffic flows through a LiteLLM proxy, the gateway can be your entire Wayfound integration — no SDK, no application code changes. Point the gateway's OpenTelemetry callback at Wayfound and every virtual key that sends traffic appears in **Settings → AI Gateway**, with its alias, team, and traffic stats. Flip **Supervise** on a key and Wayfound creates its Supervisor, connects it, and backfills roughly the last week of that key's traffic — analyzed sessions appear within minutes. ### Prerequisites 1. A LiteLLM proxy you operate (the standard deployment with `config.yaml`). 2. A Wayfound **API key** (Settings → API Keys). You do **not** need to create agents in Wayfound first — the Supervise toggle creates and publishes the agent for you. ### 1. Point the gateway at Wayfound In the proxy's `config.yaml`: ```yaml litellm_settings: callbacks: ["otel"] ``` And in the gateway's environment: ```bash OTEL_EXPORTER="otlp_http" OTEL_ENDPOINT="https://app.wayfound.ai/api/otel/v1/traces" OTEL_HEADERS="Authorization=Bearer " ``` Note the literal space in `Bearer ` — LiteLLM parses its header env vars itself without URL-decoding, so the `%20` encoding used with standard OTel SDK exporters would be sent verbatim and fail authentication. Restart the proxy after the change. This is a one-time setup by whoever operates the gateway. ### 2. Supervise your keys Most teams already issue one virtual key per application for cost tracking — if that's you, the key *is* the agent's identity. Once traffic flows, open **Settings → AI Gateway** in Wayfound: * Every key seen in gateway traffic is listed with its alias, team, first/last seen, and traffic volume. * Toggle **Supervise** on a key to create its Supervisor and backfill recent traffic. New traffic routes to it automatically — no per-request metadata needed. * Key rotation and alias renames are detected and handled automatically. ### 3. Group multi-turn conversations One request is one trace. To make a multi-turn conversation appear as a single Wayfound session, pass LiteLLM's session metadata on each completion request: ```json { "model": "...", "metadata": { "session_id": "chat-123" }, "messages": [...] } ``` Requests sharing a `session_id` merge into one ordered session; requests without one become single-turn sessions. (The LiteLLM dashboard playground sends no metadata, so playground messages always land as separate single-turn sessions.) ### 4. Multiple agents behind one key If several agents share a single virtual key, add per-request metadata to route each request to its own Supervisor — it overrides the key's default agent: ```json { "metadata": { "wayfound_agent_id": "" } } ``` ### What you get — and what needs instrumentation Gateway-only supervision captures every LLM call's user and assistant messages, model, token counts, latency, and errors, grouped into per-agent sessions with full Supervisor analysis. What it can't see: tool *executions* as structured events, or agent/handoff structure inside your application (tool calls appear only as text within messages). For that, add framework instrumentation — see Sending OpenTelemetry Traces to Wayfound. Instrumented apps and the gateway work together: Wayfound de-duplicates automatically, with the framework's spans as the source of truth and the gateway's filling in token and cost details. ### Troubleshooting * **No keys appearing in Settings → AI Gateway** — confirm the `otel` callback is in `config.yaml`, the `OTEL_*` variables are set in the gateway's environment (not just your shell), and the proxy was restarted. Check the header uses a literal space, not `%20`. * **HTTP 403 from the endpoint** — the Wayfound API key is missing or invalid, or OTLP ingestion isn't enabled for your organization. * **Every request is its own session** — requests aren't sending `metadata.session_id` (see above). * **Sessions on the wrong agent** — a shared key without `metadata.wayfound_agent_id` routes everything to the key's default agent. ### Notes and limits * Endpoint: OTLP/HTTP only (protobuf or JSON, gzip OK). gRPC is not supported. * Metrics and logs are accepted and discarded — only traces are processed. * Max request size: 4.5 MB per export batch. * Sessions appear shortly after a trace goes quiet (roughly a minute), not instantly — spans are assembled asynchronously. * Backfill on Supervise covers roughly the last 7 days of staged traffic. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.wayfound.ai/ai-gateways/connecting-litellm-gateway.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.