0 Fork 0

Files

T

haylanandClaude Bot a0e5e9704b docs: restructure around source-of-truth pages

- CONTEXT.md glossary
- README reduced to intro, quickstart and doc index
- new architecture.md, gateway.md (from proxy-key-onboarding.md),
  operations.md; network-access.md rewritten
- coding-cli-setup: stale OMNIROUTE_PORT:-4000 endpoint, terms and
  duplicated per-slot/classifier explanations fixed
- research/README.md index; proxy-request-priority.md moved there as
  superseded

Every doc names keys instead of restating values and links its source.

Refs #55 (#56, #57, #59, #60)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

2026-09-24 21:51:01 +02:00

1.8 KiB

Raw Blame History

LLM-Server

A self-hosted inference stack: local models served on one GPU machine, reached by other tools through a single authenticated entry point. This file is a glossary only — for how things are configured, follow the links in the docs to the source.

Language

Machine

Server: The one machine that runs the stack. Avoid: AI box, R9700 box, this machine

Host: The Docker host, as opposed to a container — only used for that distinction. Avoid: using it as a synonym for Server

Serving models

Backend: A model server process that serves one model to the gateway (llama-server, qwen-classifier). Avoid: provider, model server

Slot: One concurrent request lane inside a backend; a backend handles as many requests at once as it has slots. Avoid: parallel request, lane

Main model: The large model that does the actual work for workloads. Avoid: chat model

Classifier model: The small model that judges a coding CLI's tool calls before they run. Avoid: fast model (fastModel only when quoting qwen-code's config key)

Reaching the models

Gateway: The single entry point every model call goes through (OmniRoute). Avoid: proxy, AI proxy

Provider connection: The gateway's registration of one backend. Avoid: provider (alone), connection (alone)

API key: A credential issued by the gateway to exactly one workload. Avoid: virtual key, workload key

Workload: Anything that calls the gateway with its own API key. Avoid: client, consumer

Coding CLI: A workload that is an interactive coding agent (Claude Code, Kimi CLI, OpenCode, Qwen Code). Avoid: harness

Reverse proxy: The existing Nginx Proxy Manager instance that publishes the gateway under hostnames; not part of this stack. Avoid: proxy (alone)