Merge branch 'main' into feat-rag-databases
This commit is contained in:
@@ -105,34 +105,6 @@ COMFYUI_PGID=
|
||||
HOST_VIDEO_GID=
|
||||
HOST_RENDER_GID=
|
||||
|
||||
# --- llama.cpp / fast model (second, always-resident instance — see
|
||||
# docs/research/fast-model-choice.md and issue #44) ---
|
||||
# Qwen3-4B-Instruct-2507: architecturally non-thinking (never emits
|
||||
# <think> blocks, unlike Qwen3-1.7B/0.6B which need a per-call toggle) —
|
||||
# picked specifically so it stays fast enough for qwen-code's Auto Mode
|
||||
# classifier (Stage 1 wants ~300ms). Same publisher (unsloth) as the main
|
||||
# model for consistency.
|
||||
LLAMA_FAST_MODEL_FILE=Qwen3-4B-Instruct-2507-UD-Q8_K_XL.gguf
|
||||
# Same reasoning as LLAMA_GPU_LAYERS above — full GPU offload, this model
|
||||
# is dense too.
|
||||
LLAMA_FAST_GPU_LAYERS=999
|
||||
# --ctx-size is the TOTAL across every LLAMA_FAST_PARALLEL slot, not per
|
||||
# request — same halving already called out for the main model above.
|
||||
# Was PARALLEL=2, silently halving this to 4096/slot — too small: a real
|
||||
# classifier call (hints + environment + recent tool-call history) hit
|
||||
# "exceeds the available context size (4096 tokens)" in practice, which
|
||||
# qwen-code surfaces as "Auto Mode couldn't classify this action
|
||||
# (Classifier stage 1 unavailable)" — see issue #5. Fixed by dropping to
|
||||
# a single slot instead of raising ctx-size (no extra VRAM, and this
|
||||
# service doesn't need concurrent classifier calls the way the main
|
||||
# model needs concurrent chat sessions) — the full 8192 now goes to the
|
||||
# one slot. If hints.allow/softDeny/hardDeny ever approach their
|
||||
# 50-entries-each ceiling, raise LLAMA_FAST_CTX_SIZE instead — qwen-code
|
||||
# caps those at 200 chars x 150 entries plus 40,000 chars of
|
||||
# historical-action context, which can exceed 8192 tokens worst-case.
|
||||
LLAMA_FAST_CTX_SIZE=8192
|
||||
LLAMA_FAST_PARALLEL=1
|
||||
|
||||
# --- ComfyUI diffusion model (Qwen-Image, FP8 — see docs/research/
|
||||
# image-generation-model-choice.md and issue #42) ---
|
||||
# Three files: diffusion weights, text encoder, VAE — all from the official
|
||||
|
||||
Reference in New Issue
Block a user