fix(litellm): default max_tokens=4096 for the reasoning model #20

2026-09-02T18:55:46Z

haylan commented

2026-09-02 18:55:46 +00:00

Qwen3 spends output tokens on reasoning_content before writing content. Without an explicit max_tokens, requests hit llama.cpp's low default and the model runs out mid-thought — finish_reason=length, empty content, no reply in Open WebUI.

Sets litellm_params.max_tokens: 4096 as a default (not a cap) on the model in litellm-config.yaml, so every caller gets a sane budget unless it overrides.

🤖 Generated with Claude Code

Qwen3 spends output tokens on reasoning_content before writing content. Without an explicit max_tokens, requests hit llama.cpp's low default and the model runs out mid-thought — finish_reason=length, empty content, no reply in Open WebUI. Sets litellm_params.max_tokens: 4096 as a default (not a cap) on the model in litellm-config.yaml, so every caller gets a sane budget unless it overrides. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

haylan added 1 commit 2026-09-02 18:55:46 +00:00

fix(litellm): default max_tokens=4096 for the reasoning model 100fed4274

Qwen3 spends output tokens on reasoning_content before writing content.
Open WebUI's default chat request doesn't set max_tokens, so it fell
through to llama.cpp's low default and the model ran out mid-thought,
returning finish_reason=length with empty content — no reply shown in
Open WebUI. Confirmed via a manual /v1/chat/completions call: works with
max_tokens=2000, fails without it.

litellm_params.max_tokens is a default, not a cap — any caller (or Open
WebUI's per-model Advanced Params) that sets its own max_tokens still
overrides it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q2CR8yawSf7pwAVYnjwFea

haylan merged commit 949802fb2b into main

2026-09-02 18:58:12 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: haylan/LLM-Server#20