docs(research): add inference engine, spec decoding and multi-GPU research
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
a89cccb3d5
commit
1484907ff8
2 files changed
+145
No files matched your search
@@ -27,6 +27,7 @@ Each file here is a **dated snapshot**: research or a decision record as it stoo
|
||||
| [proxy-request-priority.md](proxy-request-priority.md) | Superseded by the OmniRoute migration | Priority tiers via LiteLLM's scheduler, never built on OmniRoute |
|
||||
| [proxy-shadow-pricing.md](proxy-shadow-pricing.md) | Superseded by the OmniRoute migration | Shadow cost was wired into LiteLLM config, and nothing implements it now |
|
||||
| [proxy-tool-choice.md](proxy-tool-choice.md) | Superseded by the OmniRoute migration | Chose LiteLLM. It mentions Open WebUI, which has been removed |
|
||||
| [inference-engine-spec-decoding-multi-gpu.md](inference-engine-spec-decoding-multi-gpu.md) | Current | llama.cpp vs. vLLM vs. SGLang on R9700: drafters, multi-GPU, licenses, `--cache-ram` misspelling bug. Not benchmarked |
|
||||
| [qwen3.8-27b-quant.md](qwen3.8-27b-quant.md) | Current | Main model quant and VRAM budget |
|
||||
| [qwen3.8-27b-throughput-tuning.md](qwen3.8-27b-throughput-tuning.md) | Current | Tokens/s tuning from the server log, DDR RAM as a cache tier, comparison with five outside R9700 configs. Proposals only, nothing applied |
|
||||
| [qwen3.8-27b-tool-calling.md](qwen3.8-27b-tool-calling.md) | Current | Tool-calling parser risk for the main model |
|
||||
|
||||
Reference in new issue
Block a user