haylan/LLM-Server
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity
168 Commits 1 Branch 0 Tags
main
Commit Graph
5 Commits
Author SHA1 Message Date
haylanandClaude Bot 8927e440aa feat(llm): make spec decoding and KV cache type .env toggles 
Move all speculative-decoding flags into LLAMA_SPEC_ARGS (blank = off)
and the KV cache quant into LLAMA_KV_CACHE_TYPE, so token-corruption
A/B tests are a single .env edit instead of a compose edit.
Replaces LLAMA_DRAFT_MODEL_FILE and LLAMA_SPEC_N_MAX.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-31 18:22:44 +01:00
haylanandClaude Bot 7d142ed59d fix(models): correct DFlash2 Q8_0 drafter download URL 
The URL used the filename as the Hugging Face repo name. Also align
the compose fallback for LLAMA_DRAFT_MODEL_FILE with Q8_0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-31 17:01:27 +01:00
haylan e4febb5beb fix(models): update LLAMA_DRAFT_MODEL_FILE and its URL to correct variant 2026-10-31 16:59:34 +01:00
haylan d5712d5fd4 chore(models): update LLAMA_DRAFT_MODEL_FILE and its URL to use BF16 variant 2026-10-31 16:54:15 +01:00
haylanandClaude Bot 8144286946 refactor(models): merge downloaders into one, driven by models.yml 
Replace the per-model downloader services with a single downloader
that reads file/url pairs from models.yml. Downloads now go to a
.part file first, so an interrupted run can't leave a truncated
model that later runs skip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-31 16:48:58 +01:00