diff --git a/docs/research/connecting-cloud-providers.md b/docs/research/connecting-cloud-providers.md new file mode 100644 index 0000000..0a68b3f --- /dev/null +++ b/docs/research/connecting-cloud-providers.md @@ -0,0 +1,149 @@ +# Connecting real cloud LLM providers (Anthropic Claude et al.) behind the LiteLLM gateway + +**Date:** 2026-08-26 +**Scope:** Can the existing $20/mo Claude Pro subscription be used as a backend behind our +self-hosted LiteLLM gateway (`docker-compose.yml`, `litellm-config.yaml`) instead of, or alongside, +llama.cpp? If not, what's the concrete path to add real Anthropic (and other) cloud models? + +## 1. Claude Pro subscription quota vs. Anthropic API — is it the same thing? + +**No. They are two separate products with separate billing, and Anthropic's own documentation and +Terms of Service explicitly prohibit routing Pro/Max subscription credentials through third-party +tools like a self-hosted gateway.** + +### 1a. Pro plan does not include API access + +Anthropic's help center is explicit: + +> "The Pro plan does not include API usage through the Claude Console. If you're interested in both +> enhanced Claude features and the Claude API, you'll need to set up Console access to pay for API +> usage separately." +— https://support.claude.com/en/articles/8325606-what-is-the-pro-plan + +Pro/Max are flat-fee subscriptions to *products* (claude.ai web/desktop/mobile chat, and Claude Code) +with rolling usage limits. The Anthropic API (Console/`platform.claude.com`) is a metered, +pay-per-token product with its own separate billing account. There is no documented way to +authenticate a non-Anthropic tool against Pro/Max quota "as if" it were an API key — no such +integration exists. + +### 1b. How Claude Code itself authenticates, and why that path is closed off to other tools + +When you run `claude` and log in via `/login` with a claude.ai account, Claude Code obtains an +**OAuth token scoped to the subscription** (standard OAuth 2.0 authorization-code flow against +Anthropic's consent page). Anthropic also documents `claude setup-token`, which mints a **long-lived +(1-year) OAuth token** (`CLAUDE_CODE_OAUTH_TOKEN`, prefixed `sk-ant-oat01-...`) explicitly *for use +outside interactive login* — e.g. CI. This is real, officially documented, subscription-backed +credential material that is technically exportable as an environment variable. +— https://code.claude.com/docs/en/authentication + +**However, Anthropic's official Legal & Compliance page for Claude Code states this is authorized for +Claude Code (and other native Anthropic apps) only, not for arbitrary third-party tools:** + +> "**OAuth authentication** is intended exclusively for purchasers of Claude Free, Pro, Max, Team, and +> Enterprise subscription plans and is designed to support ordinary use of Claude Code and other +> native Anthropic applications." +> +> "Anthropic does not permit third-party developers to offer Claude.ai login into their own +> applications, or to route requests through Free, Pro, or Max plan credentials on behalf of their +> users. Moreover, developers may not collect, store, or intermediate Claude.ai credentials or session +> tokens — sign-in to a Claude account must complete through Anthropic's own flow." +> +> "Anthropic reserves the right to take measures to enforce these restrictions and may do so without +> prior notice." +— https://code.claude.com/docs/en/legal-and-compliance ("Authentication and credential use" section) + +That page cites the governing documents directly: +- Consumer Terms of Service (Free/Pro/Max): https://www.anthropic.com/legal/consumer-terms +- Commercial Terms of Service (Team/Enterprise/API): https://www.anthropic.com/legal/commercial-terms +- Anthropic Usage Policy: https://www.anthropic.com/legal/aup + +**This is not just a ToS technicality — it's actively enforced.** In March 2026 Anthropic used +server-side enforcement (in addition to legal action) to block third-party "harnesses" (e.g. +OpenClaw and similar tools) from routing traffic through Claude subscription OAuth credentials, +specifically citing that such traffic bypasses the telemetry/behavior the Claude Code harness +provides. Coverage: https://www.theregister.com/software/2026/02/20/anthropic-clarifies-ban-on-third-party-tool-access-to-claude/ +(secondary source; the primary enforcement basis is the Legal & Compliance page above, which is +current as fetched today). + +**Community finding (flagged as unofficial/against ToS):** technically, `claude setup-token`'s OAuth +token *can* be handed to LiteLLM (some LiteLLM community docs / discussions describe pointing +`ANTHROPIC_AUTH_TOKEN`/`CLAUDE_CODE_OAUTH_TOKEN` at a proxy, e.g. +https://docs.litellm.ai/docs/tutorials/claude_code_max_subscription and +https://github.com/BerriAI/litellm/discussions/30827). This works at the protocol level because the +token is a bearer credential like any other. **It is exactly the pattern Anthropic's compliance page +above says is not permitted** ("route requests through Free, Pro, or Max plan credentials on behalf +of \[other] users" / outside "ordinary use of Claude Code and other native Anthropic applications"). +Doing this for **personal, single-user use** through the unmodified `claude` binary (e.g. just running +`claude` itself pointed at your own gateway) is different from what's prohibited — the prohibition +targets *intermediating* the subscription credential through a third-party tool/product on behalf of +requests that aren't Claude Code itself. Routing arbitrary LiteLLM/OpenWebUI traffic through a +subscription OAuth token extracted from `claude setup-token` falls squarely in the prohibited +category. **Do not build this into the gateway.** + +### 1c. Bottom line on subscription reuse + +No documented, ToS-compliant way exists to point LiteLLM (or any third-party gateway) at Claude +Pro/Max quota instead of a real API key. The $20/mo subscription is for claude.ai and Claude Code +usage only. + +## 2. The real path: an Anthropic API key (Console, pay-per-token) + +This is the supported way to add Claude models to LiteLLM, and it's the same config shape already +used for llama.cpp in `litellm-config.yaml`. + +LiteLLM's Anthropic provider docs (https://docs.litellm.ai/docs/providers/anthropic) give this +`model_list` shape: + +```yaml +model_list: + - model_name: claude-sonnet-5 # whatever alias you want callers to use + litellm_params: + model: anthropic/claude-sonnet-5 # "anthropic/" tells LiteLLM which provider + api_key: os.environ/ANTHROPIC_API_KEY +``` + +Practical steps: +1. Create a Console account / org at https://platform.claude.com (separate from claude.ai login), + generate an API key there. +2. Add it to `.env` as `ANTHROPIC_API_KEY=sk-ant-api03-...` and reference it via + `os.environ/ANTHROPIC_API_KEY` in `litellm-config.yaml`, following the existing pattern used for + `LITELLM_MASTER_KEY`/`LITELLM_SALT_KEY` in `docker-compose.yml`. +3. Add a `model_list` entry per Claude model you want exposed (e.g. `claude-sonnet-5`, + `claude-haiku-4-5`), each with its own `input_cost_per_token`/`output_cost_per_token` if you want + LiteLLM's cost tracking to reflect real spend (today `litellm-config.yaml`'s cost fields are a + shadow estimate against Sonnet 5 pricing for the *local* model — real Anthropic model entries + should carry the model's *actual* published rate from + https://platform.claude.com/docs/en/about-claude/pricing). +4. **This is billed separately from, and in addition to, the $20/mo subscription** — pay-per-token, + metered on the Console account, no relationship to Pro/Max usage limits. + +No other config changes are implied — the router/`general_settings` blocks already in place don't +need to change for a second provider; LiteLLM routes per-`model_name` based on the `litellm_params` +each entry declares. + +## 3. Other providers — same general pattern + +Every major consumer AI subscription (OpenAI's ChatGPT Plus, and by the same logic Google's Gemini +subscription tiers) works identically to Anthropic's: **the consumer chat subscription and the +pay-per-token developer API are separate products with separate billing, and the subscription does +not unlock API access.** Confirmed for OpenAI: "ChatGPT Plus does not include API access... no API +key in the box, no monthly API credits, and no discount on per-token prices. The OpenAI API is a +separate product with its own billing." There's no equivalent OAuth-token-reuse loophole documented +for OpenAI/Google either — this isn't an Anthropic-specific restriction, it's the standard shape of +how these companies segment consumer vs. developer products. Adding OpenAI, Gemini, or any other +cloud model to LiteLLM means the same recipe as Anthropic: get a real pay-per-token API key from that +provider's own developer console and add a `model_list` entry with `litellm_params.model` set to +that provider's LiteLLM prefix (`openai/...`, `gemini/...`, etc.) — see +https://docs.litellm.ai/docs/providers for the full prefix list. + +## 4. Recommended next step + +- Do **not** attempt to feed the Claude Pro subscription's OAuth token into LiteLLM/Open WebUI — it's + against Anthropic's Consumer Terms of Service and Usage Policy, and Anthropic has shown it will + enforce this server-side without notice (per the March 2026 crackdown on third-party harnesses). +- If real cloud Claude models behind the gateway are wanted, get a **separate Anthropic Console API + key** (pay-per-token, its own bill, independent of the $20/mo subscription) and add it as a + `model_list` entry per §2 above. Same recipe for any other provider (§3). +- Keep using the existing $20/mo Claude Pro subscription only for what it's licensed for: the claude.ai + web app and the Claude Code CLI itself (this session included) — not as a backend behind the + self-hosted gateway.