docs(research): add qwen3.8-27b decode speed analysis
- Add research note on decode tokens/s versus context depth for Qwen3.8-27B + DFlash2 on the R9700 - Covers draft length (5, not 7), acceptance falling with depth, bandwidth ceiling, and an ordered experiment plan. Proposals only, nothing applied - Index the note in docs/research/README.md - Also stages .serena/ project config and llmserver.log
This commit is contained in:
1 parent
5b866b93ff
commit
e5cc2a6360
5 files changed
+10416
No files matched your search
@@ -29,6 +29,7 @@ Each file here is a **dated snapshot**: research or a decision record as it stoo
|
||||
| [proxy-shadow-pricing.md](proxy-shadow-pricing.md) | Superseded by the OmniRoute migration | Shadow cost was wired into LiteLLM config, and nothing implements it now |
|
||||
| [proxy-tool-choice.md](proxy-tool-choice.md) | Superseded by the OmniRoute migration | Chose LiteLLM. It mentions Open WebUI, which has been removed |
|
||||
| [inference-engine-spec-decoding-multi-gpu.md](inference-engine-spec-decoding-multi-gpu.md) | Current | llama.cpp vs. vLLM vs. SGLang on R9700: drafters, multi-GPU, licenses, `--cache-ram` misspelling bug. Not benchmarked |
|
||||
| [qwen3.8-27b-decode-speed.md](qwen3.8-27b-decode-speed.md) | Current | Decode tokens/s: why DFlash2 acceptance and speed fall with depth (draft length is 5, not 7), n_max/p_min, KV and slot costs, bandwidth ceiling, ordered experiment plan. Builds on the throughput-tuning note. Proposals only, nothing applied |
|
||||
| [qwen3.8-27b-quant.md](qwen3.8-27b-quant.md) | Current | Main model quant and VRAM budget |
|
||||
| [qwen3.8-27b-throughput-tuning.md](qwen3.8-27b-throughput-tuning.md) | Current | Tokens/s tuning from the server log, DDR RAM as a cache tier, comparison with five outside R9700 configs. Proposals only, nothing applied |
|
||||
| [qwen3.8-27b-tool-calling.md](qwen3.8-27b-tool-calling.md) | Current | Tool-calling parser risk for the main model |
|
||||
|
||||
Reference in new issue
Block a user