Migrate Open WebUI (and existing consumers) to route through the new proxy #15

New Issue

2026-08-25T04:40:06Z

haylan commented

2026-08-25 04:40:06 +00:00

Part of #9

Question

Point Open WebUI's model connection (and any coding-CLI configs from map #1's #6) at the new proxy service from #14 instead of llama.cpp directly, using a provisioned key per #12's workflow. Confirm nothing regresses (chat, RAG/Memory via Qdrant, tool-calling for the coding CLIs).

Part of #9 ## Question Point Open WebUI's model connection (and any coding-CLI configs from map #1's #6) at the new proxy service from #14 instead of llama.cpp directly, using a provisioned key per #12's workflow. Confirm nothing regresses (chat, RAG/Memory via Qdrant, tool-calling for the coding CLIs).

haylan added the wayfinder:task label 2026-08-25 04:40:06 +00:00

haylan added a new dependency 2026-08-25 04:40:15 +00:00

#14 Author the docker-compose service for the chosen proxy

haylan referenced this issue

2026-08-25 04:40:25 +00:00

AI gateway/proxy: routing, per-workload keys, and cloud-cost estimation for the local LLM stack #9

haylan referenced this issue from a commit

2026-08-25 05:11:17 +00:00

Author LiteLLM AI proxy service (resolves #14)

haylan referenced this issue

2026-08-25 05:11:23 +00:00

Author the docker-compose service for the chosen proxy #14

haylan self-assigned this 2026-08-25 05:12:51 +00:00

haylan referenced this issue from a commit

2026-08-25 05:18:04 +00:00

Migrate Open WebUI and coding CLIs to the AI proxy (resolves #15)

haylan closed this issue

2026-08-25 05:18:04 +00:00

haylan commented

2026-08-25 05:18:10 +00:00

Resolved: migrated Open WebUI and the coding CLIs to the proxy.

Open WebUI's OPENAI_API_BASE_URL now points at litellm:4000 instead of llama-server:8080, using a provisioned virtual key (OPENWEBUI_LITELLM_KEY) instead of the old unchecked dummy key. llama-server's host port publish is dropped entirely — it's internal-only on the ai-stack docker network now, reached only via litellm, since the proxy is the sole intended entry point (llama.cpp's own API was always unauthenticated). docs/coding-cli-setup.md rewritten: all three CLIs (Claude Code, Kimi, OpenCode) now point at the proxy's endpoints (port 4000) with their own per-CLI virtual key, per docs/proxy-key-onboarding.md.

Committed on main.

Resolved: migrated Open WebUI and the coding CLIs to the proxy. Open WebUI's `OPENAI_API_BASE_URL` now points at `litellm:4000` instead of `llama-server:8080`, using a provisioned virtual key (`OPENWEBUI_LITELLM_KEY`) instead of the old unchecked dummy key. llama-server's host port publish is dropped entirely — it's internal-only on the ai-stack docker network now, reached only via litellm, since the proxy is the sole intended entry point (llama.cpp's own API was always unauthenticated). docs/coding-cli-setup.md rewritten: all three CLIs (Claude Code, Kimi, OpenCode) now point at the proxy's endpoints (port 4000) with their own per-CLI virtual key, per docs/proxy-key-onboarding.md. Committed on main.

haylan referenced this issue from a commit

2026-09-02 20:55:45 +00:00

fix(lazytainer): drop network_mode: host, join ai-stack (fixes self-ID panic)

haylan referenced this issue

2026-09-03 04:19:22 +00:00

opencode: context size and compaction settings for the litellm/qwen3.8-27b-local stack #26

haylan referenced this issue

2026-09-03 04:19:56 +00:00

Point opencode's provider block at litellm.home with a virtual key #29

Sign in to join this conversation.