haylanandClaude Bot
51684fb7a4
DFlash2 truncates its draft when selector confidence drops below p_min, cutting wasted verify work on long, low-acceptance thinking output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LLM-Server
Source of truth:
docker-compose.yml(what runs) andscripts/update.sh(how it's deployed). The docs explain why; if they disagree with the source, the source wins.
A self-hosted inference stack on one AMD GPU server. A llama.cpp backend serving the main model sits behind an OmniRoute gateway. Every workload, mostly coding CLIs, reaches it through the gateway with its own API key. Terms are defined in CONTEXT.md.
Quickstart
On the server:
./scripts/update.sh
It creates and syncs .env, generates secrets, downloads the models, and brings the stack up. It's safe to re-run at any time. What it does and why: docs/operations.md.
Docs
| Doc | What it covers |
|---|---|
CONTEXT.md |
Glossary: the terms every doc uses |
docs/architecture.md |
Each service, why it exists, and why it's configured the way it is |
docs/gateway.md |
OmniRoute: dashboard, provider connections, API keys, timeouts, web search |
docs/network-access.md |
Who can reach which port, and why |
docs/operations.md |
What update.sh does: env sync, secrets, downloads; speed bench |
docs/coding-cli-setup/ |
Pointing Claude Code, Kimi CLI, OpenCode and Qwen Code at the gateway |
docs/research/ |
Dated research and decision records, each marked current or superseded |
docs/server-planing.md |
Background research: a 4× GPU rack build. Not an active plan |
docs/agents/ |
Instructions for coding agents: issue tracker, domain docs, writing docs |