Fix llama-server-fast context-size exhaustion breaking Auto Mode #49

Merged
haylan merged 3 commits from fix-fastmodel-context-size into main 2026-09-06 20:21:53 +00:00
3 Commits
Author SHA1 Message Date
haylan ea7b05fb99 feat: interactive per-key conflict resolution in update.sh's config sync
Replaces the previous commit's blind force-overwrite with a real
choice, per user feedback: force-overwriting server config without
asking was the wrong default.

- A tracked config value (real default in .env.example) that already
  matches .env is left alone silently — no prompt, no noise.
- A value that DIFFERS is a conflict, shown on one screen (all
  conflicts together, not one prompt per key) via gum
  (charmbracelet/gum) — single static binary fetched as a release
  tarball into .cache/gum/ (gitignored), no build step, no package
  manager dependency. Falls back to a plain read-based prompt if gum
  can't be fetched (offline, unsupported arch).
- Non-interactive (no TTY — cron, CI, piped): any conflict is a hard
  error (exit 1, lists every conflicting key) unless --force is
  passed, which accepts every new value automatically — matches how
  this PR's own fix needs to land unattended.
- Secrets and host-resolved values are completely unaffected either
  way — untouched by this loop, same as before.

Verified in an isolated sandbox against the exact scenario from this
PR (stale LLAMA_FAST_PARALLEL=2 vs git's 1):
- no TTY, no --force: exits 1, prints the diff, doesn't touch .env
- no TTY, --force: LLAMA_FAST_PARALLEL corrected 2 -> 1, an
  OMNIROUTE secret confirmed untouched (not regenerated)
docker compose config -q still passes.

Not verified: the interactive gum path itself (needs a real TTY,
couldn't allocate a pty in this sandbox) — worth confirming for real
on the server, including that gum's release asset naming actually
matches what ensure_gum() expects.

Refs #5
2026-09-06 22:20:27 +02:00
haylan 9def240a8e feat: update.sh force-syncs tracked config from .env.example
Follows directly from the previous commit's caveat: this PR's own fix
(LLAMA_FAST_PARALLEL=2 -> 1) wouldn't have taken effect on the server
without a manual .env edit, because set_if_blank never touches an
already-set value — by design, for secrets, but the same logic was
silently protecting stale copies of ordinary tunable config too.

Every KEY=VALUE line in .env.example with a real (non-blank) default
is now force-synced into .env on every run. Secrets and host-resolved
values are unaffected — .env.example already leaves those blank on
purpose, so the sync loop naturally skips them and they keep going
through set_if_blank as before.

Trade-off, called out in both the script's header and the sync loop's
own comment: there's no such thing as a persistent server-only
override for these keys anymore — a hand-edited value not reflected
in git gets reverted on the next run. That's the intended behavior.

Verified against a simulated stale .env matching the real scenario
from this PR: LLAMA_FAST_PARALLEL correctly overwritten 2 -> 1, an
OMNIROUTE secret left untouched. bash -n and docker compose config -q
both pass.

Refs #5
2026-09-06 22:10:51 +02:00
haylanandClaude-Bot 52a92f6508 fix: llama-server-fast context-size exhaustion breaking Auto Mode classifier
Real failure: "Auto Mode couldn't classify this action (Classifier
stage 1 unavailable)". Reproduced directly against the server:

  {"error":{"message":"[400]: request (6186 tokens) exceeds the
  available context size (4096 tokens)"...

LLAMA_FAST_CTX_SIZE=8192 is the TOTAL across every LLAMA_FAST_PARALLEL
slot, not per-request — the main model's own .env.example comment
already calls this out, missed it when llama-server-fast was set up
(#44). With PARALLEL=2 that's 4096/slot, too small for a real
classifier call (hints + environment + recent tool-call history).

Fixed by dropping to a single slot (LLAMA_FAST_PARALLEL=1) rather than
raising ctx-size — this service doesn't need concurrent classifier
calls the way the main model needs concurrent chat sessions, so this
costs no extra VRAM. The full 8192 now goes to the one slot.

docker compose config -q validated.

Refs #5

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MrnMEdzeQzqZE5soVEXPCx
2026-09-06 22:06:59 +02:00