haylanandClaude Bot
a0e5e9704b
- CONTEXT.md glossary - README reduced to intro, quickstart and doc index - new architecture.md, gateway.md (from proxy-key-onboarding.md), operations.md; network-access.md rewritten - coding-cli-setup: stale OMNIROUTE_PORT:-4000 endpoint, terms and duplicated per-slot/classifier explanations fixed - research/README.md index; proxy-request-priority.md moved there as superseded Every doc names keys instead of restating values and links its source. Refs #55 (#56, #57, #59, #60) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1.9 KiB
1.9 KiB
LLM-Server
Source of truth:
docker-compose.yml(what runs) andscripts/update.sh(how it's deployed). The docs explain why; if they disagree with the source, the source wins.
A self-hosted inference stack on one AMD GPU server. Two llama.cpp backends, the main model and the classifier model, sit behind an OmniRoute gateway. Every workload, mostly coding CLIs, reaches them through the gateway with its own API key. Terms are defined in CONTEXT.md.
Quickstart
On the server:
./scripts/update.sh
It creates and syncs .env, generates secrets, downloads the models, and brings the stack up. It's safe to re-run at any time. What it does and why: docs/operations.md.
Docs
| Doc | What it covers |
|---|---|
CONTEXT.md |
Glossary: the terms every doc uses |
docs/architecture.md |
Each service, why it exists, and why it's configured the way it is |
docs/gateway.md |
OmniRoute: dashboard, provider connections, API keys, timeouts, web search |
docs/network-access.md |
Who can reach which port, and why |
docs/operations.md |
What update.sh does: env sync, secrets, downloads |
docs/coding-cli-setup/ |
Pointing Claude Code, Kimi CLI, OpenCode and Qwen Code at the gateway |
docs/research/ |
Dated research and decision records, each marked current or superseded |
docs/server-planing.md |
Background research: a 4× GPU rack build. Not an active plan |
docs/agents/ |
Instructions for coding agents: issue tracker, domain docs, writing docs |