haylanandClaude-Bot d984c10835 feat: add llama-server-fast, a small non-thinking classifier model ...

Second, always-resident llama.cpp instance (Qwen3-4B-Instruct-2507,
Q8_0 GGUF, ~5GB VRAM) alongside the existing Qwen3.8-27B instance, for
use as qwen-code CLI's Auto Mode classifier fastModel. Model choice
researched in docs/research/fast-model-choice.md: architecturally
non-thinking (unlike Qwen3-1.7B/0.6B), --reasoning off added
defensively per a known (closed) llama.cpp misdetection bug.

- docker-compose.yml: llama-server-fast + downloader-fast services,
  omniroute depends_on updated
- .env.example: LLAMA_FAST_* vars
- scripts/update.sh: runs the new downloader profile

Refs #44

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MrnMEdzeQzqZE5soVEXPCx

2026-09-06 20:24:59 +02:00

..

connecting-cloud-providers.md

research: Claude Pro subscription vs Anthropic API for gateway backends

2026-08-26 08:51:38 +02:00

dashscope-websearch-selfhost-options.md

Create dashscope-websearch-selfhost-options.md

2026-09-06 16:10:26 +02:00

fast-model-choice.md

feat: add llama-server-fast, a small non-thinking classifier model

2026-09-06 20:24:59 +02:00

image-generation-model-choice.md

docs(research): recommend Qwen-Image FP8 for full-VRAM diffusion build

2026-09-05 21:03:13 +02:00

image-generation-options.md

docs(research): add image-gen backend survey and omniroute/qwen websearch notes

2026-09-05 21:24:23 +02:00

langchain-pgvector-vs-litellm-pgvector.md

docs(research): LangChain+pgvector-direct vs litellm-pgvector for issue #25

2026-09-02 22:02:58 +02:00

lazytainer-omniroute-idle-stop.md

docs(research): confirm lazytainer/omniroute idle-stop conflict (#40)

2026-09-05 20:57:24 +02:00

litellm-knowledgebase.md

docs(research): investigate LiteLLM knowledgebase/vector_store feature for #23

2026-09-02 21:26:39 +02:00

litellm-searxng-search.md

research: LiteLLM web-search + SearXNG wiring (issue #22)

2026-09-02 21:25:30 +02:00

omniroute-qwen-websearch.md

docs(research): add image-gen backend survey and omniroute/qwen websearch notes

2026-09-05 21:24:23 +02:00

opencode-cli-setup.md

research: OpenCode CLI install + local llama.cpp provider config

2026-08-24 18:17:14 +02:00

proxy-shadow-pricing.md

research: reference cloud pricing for LiteLLM shadow-cost estimate

2026-08-25 06:49:38 +02:00

proxy-tool-choice.md

research: self-hosted AI gateway/proxy tool choice — recommend LiteLLM

2026-08-25 06:45:10 +02:00

qwen3.8-27b-quant.md

docs: research Qwen3.8-27B Unsloth dynamic GGUF quant + VRAM budget

2026-08-24 12:16:42 +02:00

qwen3.8-27b-tool-calling.md

research: Qwen3.8-27B tool-calling compat with llama.cpp Anthropic shim

2026-08-24 12:16:09 +02:00

voidllm-evaluation.md

research: evaluate voidllm as a LiteLLM proxy replacement

2026-08-26 08:42:23 +02:00