Qwen3 spends output tokens on reasoning_content before writing content.
Open WebUI's default chat request doesn't set max_tokens, so it fell
through to llama.cpp's low default and the model ran out mid-thought,
returning finish_reason=length with empty content — no reply shown in
Open WebUI. Confirmed via a manual /v1/chat/completions call: works with
max_tokens=2000, fails without it.
litellm_params.max_tokens is a default, not a cap — any caller (or Open
WebUI's per-model Advanced Params) that sets its own max_tokens still
overrides it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q2CR8yawSf7pwAVYnjwFea
Open WebUI now points at litellm instead of llama-server directly, using a
provisioned virtual key. llama-server's host port is dropped (internal-only
on the ai-stack network) since the proxy is the only intended entry point
now. docs/coding-cli-setup.md repointed at the proxy's endpoints/ports with
per-CLI virtual keys instead of the old shared dummy key.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds litellm + litellm-db to docker-compose.yml, litellm-config.yaml with
custom shadow-cost pricing (Claude Sonnet 5 reference, per #11) and a
priority-scheduling stub (per #16, needs real-hardware smoke test), and
required LITELLM_MASTER_KEY/SALT_KEY/DB_PASSWORD env vars. Untested on real
hardware — that's #17. Open WebUI/coding CLIs still talk to llama.cpp
directly, migration is #15.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
haylanandClaude-Bot