haylan
e5cc2a6360
- Add research note on decode tokens/s versus context depth for Qwen3.8-27B + DFlash2 on the R9700 - Covers draft length (5, not 7), acceptance falling with depth, bandwidth ceiling, and an ordered experiment plan. Proposals only, nothing applied - Index the note in docs/research/README.md - Also stages .serena/ project config and llmserver.log
LLM-Server
Source of truth:
docker-compose.yml(what runs) andscripts/update.sh(how it's deployed). The docs explain why; if they disagree with the source, the source wins.
A self-hosted inference stack on one AMD GPU server. A llama.cpp backend serving the main model sits behind an OmniRoute gateway. Every workload, mostly coding CLIs, reaches it through the gateway with its own API key. Terms are defined in CONTEXT.md.
Quickstart
On the server:
./scripts/update.sh
It creates and syncs .env, generates secrets, downloads the models, and brings the stack up. It's safe to re-run at any time. What it does and why: docs/operations.md.
Docs
| Doc | What it covers |
|---|---|
CONTEXT.md |
Glossary: the terms every doc uses |
docs/architecture.md |
Each service, why it exists, and why it's configured the way it is |
docs/gateway.md |
OmniRoute: dashboard, provider connections, API keys, timeouts, web search |
docs/network-access.md |
Who can reach which port, and why |
docs/operations.md |
What update.sh does: env sync, secrets, downloads; speed bench |
docs/coding-cli-setup/ |
Pointing Claude Code, Kimi CLI, OpenCode and Qwen Code at the gateway |
docs/research/ |
Dated research and decision records, each marked current or superseded |
docs/server-planing.md |
Background research: a 4× GPU rack build. Not an active plan |
docs/agents/ |
Instructions for coding agents: issue tracker, domain docs, writing docs |