--no-cache-idle-slots is the llama.cpp#27148 mitigation for --kv-unified,
which was the cause of misspelled output with --cache-ram on.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DFlash2 truncates its draft when selector confidence drops below p_min,
cutting wasted verify work on long, low-acceptance thinking output.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move all speculative-decoding flags into LLAMA_SPEC_ARGS (blank = off)
and the KV cache quant into LLAMA_KV_CACHE_TYPE, so token-corruption
A/B tests are a single .env edit instead of a compose edit.
Replaces LLAMA_DRAFT_MODEL_FILE and LLAMA_SPEC_N_MAX.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the per-model downloader services with a single downloader
that reads file/url pairs from models.yml. Downloads now go to a
.part file first, so an interrupted run can't leave a truncated
model that later runs skip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Startup log: 'cache_reuse is not supported by this context, it will be disabled'.
Correct architecture.md, which claimed it fixed the SSE timeout.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Log shows 253 prompt-cache evictions and 146 full ~42k re-prefills at the default budget.
Add LLAMA_CACHE_RAM and the throughput-tuning research note.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Replace MTP with draft-dflash and a separate DFlash2 drafter GGUF.
Remove the qwen-classifier backend to free VRAM (GPU was at 100%).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Explanations from docker-compose.yml, .env.example and
scripts/update.sh now live in docs/architecture.md, gateway.md,
network-access.md and operations.md. Code keeps `# see docs/...`
pointers, one-line warnings (storage key, SSRF guard, WAL grace period,
git pull ordering) and "auto-filled — leave blank" markers.
Refs #55 (comment destination map #61)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- CONTEXT.md glossary
- README reduced to intro, quickstart and doc index
- new architecture.md, gateway.md (from proxy-key-onboarding.md),
operations.md; network-access.md rewritten
- coding-cli-setup: stale OMNIROUTE_PORT:-4000 endpoint, terms and
duplicated per-slot/classifier explanations fixed
- research/README.md index; proxy-request-priority.md moved there as
superseded
Every doc names keys instead of restating values and links its source.
Refs #55 (#56, #57, #59, #60)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
haylanandClaude Bot