LLM-Server

Source of truth: docker-compose.yml (what runs) and scripts/update.sh (how it's deployed). The docs explain why; if they disagree with the source, the source wins.

A self-hosted inference stack on one AMD GPU server. Two llama.cpp backends, the main model and the classifier model, sit behind an OmniRoute gateway. Every workload, mostly coding CLIs, reaches them through the gateway with its own API key. Terms are defined in CONTEXT.md.

Quickstart

On the server:

./scripts/update.sh

It creates and syncs .env, generates secrets, downloads the models, and brings the stack up. It's safe to re-run at any time. What it does and why: docs/operations.md.

Docs

Doc	What it covers
CONTEXT.md	Glossary: the terms every doc uses
docs/architecture.md	Each service, why it exists, and why it's configured the way it is
docs/gateway.md	OmniRoute: dashboard, provider connections, API keys, timeouts, web search
docs/network-access.md	Who can reach which port, and why
docs/operations.md	What update.sh does: env sync, secrets, downloads
docs/coding-cli-setup/	Pointing Claude Code, Kimi CLI, OpenCode and Qwen Code at the gateway
docs/research/	Dated research and decision records, each marked current or superseded
docs/server-planing.md	Background research: a 4× GPU rack build. Not an active plan
docs/agents/	Instructions for coding agents: issue tracker, domain docs, writing docs