docs(research): add qwen3.8-27b decode speed analysis

- Add research note on decode tokens/s versus context depth for Qwen3.8-27B + DFlash2 on the R9700
- Covers draft length (5, not 7), acceptance falling with depth, bandwidth ceiling, and an ordered experiment plan. Proposals only, nothing applied
- Index the note in docs/research/README.md
- Also stages .serena/ project config and llmserver.log
This commit is contained in:
haylan committed 2026-11-11 16:51:45 +01:00
1 parent 5b866b93ff
commit e5cc2a6360
5 files changed
+10416

No files matched your search

+1
View File
@@ -29,6 +29,7 @@ Each file here is a **dated snapshot**: research or a decision record as it stoo
| [proxy-shadow-pricing.md](proxy-shadow-pricing.md) | Superseded by the OmniRoute migration | Shadow cost was wired into LiteLLM config, and nothing implements it now |
| [proxy-tool-choice.md](proxy-tool-choice.md) | Superseded by the OmniRoute migration | Chose LiteLLM. It mentions Open WebUI, which has been removed |
| [inference-engine-spec-decoding-multi-gpu.md](inference-engine-spec-decoding-multi-gpu.md) | Current | llama.cpp vs. vLLM vs. SGLang on R9700: drafters, multi-GPU, licenses, `--cache-ram` misspelling bug. Not benchmarked |
| [qwen3.8-27b-decode-speed.md](qwen3.8-27b-decode-speed.md) | Current | Decode tokens/s: why DFlash2 acceptance and speed fall with depth (draft length is 5, not 7), n_max/p_min, KV and slot costs, bandwidth ceiling, ordered experiment plan. Builds on the throughput-tuning note. Proposals only, nothing applied |
| [qwen3.8-27b-quant.md](qwen3.8-27b-quant.md) | Current | Main model quant and VRAM budget |
| [qwen3.8-27b-throughput-tuning.md](qwen3.8-27b-throughput-tuning.md) | Current | Tokens/s tuning from the server log, DDR RAM as a cache tier, comparison with five outside R9700 configs. Proposals only, nothing applied |
| [qwen3.8-27b-tool-calling.md](qwen3.8-27b-tool-calling.md) | Current | Tool-calling parser risk for the main model |