chore: move code comments into docs, leave one-line pointers
Explanations from docker-compose.yml, .env.example and scripts/update.sh now live in docs/architecture.md, gateway.md, network-access.md and operations.md. Code keeps `# see docs/...` pointers, one-line warnings (storage key, SSRF guard, WAL grace period, git pull ordering) and "auto-filled — leave blank" markers. Refs #55 (comment destination map #61) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
a0e5e9704b
commit
05effe5c88
4 files changed
+31
-265
No files matched your search
+11
-84
@@ -1,108 +1,35 @@
|
||||
# Copy to .env and adjust — or just run ./scripts/update.sh, which creates
|
||||
# .env from this file and fills in every secret/key below it can generate
|
||||
# itself (see each var's comment). All values below are defaults baked into
|
||||
# docker-compose.yml — only uncomment/change what you actually want to
|
||||
# override.
|
||||
# Template for .env, managed by ./scripts/update.sh — see docs/operations.md#env-sync
|
||||
|
||||
# --- llama.cpp / model ---
|
||||
# --- llama.cpp / models --- see docs/architecture.md#llama-server
|
||||
LLAMA_MODEL_FILE=Qwen3.8-27B-UD-Q4_K_XL.gguf
|
||||
# 999 = every layer on GPU (this model is dense, not MoE, and already fits
|
||||
# fully in 32GB VRAM — see docs/research/qwen3.8-27b-quant.md). Lower this
|
||||
# to leave that many fewer layers on GPU and push the rest to CPU/system RAM
|
||||
# if something else is contending for VRAM — llama.cpp has no separate
|
||||
# "RAM offload" flag, --n-gpu-layers *is* the RAM-offload knob for a dense
|
||||
# model. Don't reach for --n-cpu-moe/--cpu-moe/--override-tensor "exps" —
|
||||
# those target Mixture-of-Experts models (e.g. Qwen3.8-2.4T-A95B), not this
|
||||
# one, and are no-ops here.
|
||||
# There's no separate "then SSD" tier to enable either: llama.cpp mmaps the
|
||||
# model file by default (no --no-mmap here), so if GPU+RAM ever can't hold
|
||||
# the working set, the OS pages the rest in from disk automatically — an
|
||||
# implicit, slow last resort, not a config knob. An explicit tiered SSD
|
||||
# offload has been an open llama.cpp feature request since 2025 (still
|
||||
# unimplemented): https://github.com/ggml-org/llama.cpp/discussions/12507
|
||||
LLAMA_GPU_LAYERS=999
|
||||
# 262144 = this model's true max (max_position_embeddings in Qwen/Qwen3.8-27B's
|
||||
# config.json) — the largest --ctx-size llama.cpp will even accept for it.
|
||||
# fp16 KV cache at full context would be ~16GB, on top of 17.6GB weights =
|
||||
# ~33.6GB, which does NOT fit the 32GB R9700 on its own. docker-compose.yml
|
||||
# now runs --cache-type-k/v q8_0, which roughly halves KV memory (~8GB at
|
||||
# this size) — total ~25.6GB, ~6GB headroom, the same footprint the old
|
||||
# 131072 fp16 setting used. See docs/research/qwen3.8-27b-quant.md.
|
||||
LLAMA_CTX_SIZE=262144
|
||||
# Concurrent request slots — the real hardware ceiling for this GPU, not a
|
||||
# tunable to raise for throughput (was implicitly 4, llama.cpp's compiled-in
|
||||
# default; dropped to 2 because more contended prefill was blowing requests
|
||||
# past OmniRoute's idle timeout — see OMNIROUTE_STREAM_IDLE_TIMEOUT_MS below).
|
||||
# The 3rd+ request now queues on llama.cpp itself instead — its own queue has
|
||||
# no timeout (tools/server/server-queue.cpp), it just waits for a slot — so
|
||||
# the timeout that matters moved to OmniRoute's per-connection
|
||||
# providerSpecificData.timeoutMs (dashboard/API only, not in this file; see
|
||||
# handoff notes in the issue tracker). Each slot gets LLAMA_CTX_SIZE /
|
||||
# LLAMA_PARALLEL tokens of context — real sessions have hit ~66K tokens, so
|
||||
# don't drop LLAMA_CTX_SIZE without checking that per-slot number stays
|
||||
# comfortably above observed usage.
|
||||
LLAMA_PARALLEL=2
|
||||
|
||||
# Dedicated GPU-resident backend for qwen-code's tool-call harmfulness
|
||||
# classifier (fastModel in ~/.qwen/settings.json) — see docker-compose.yml's
|
||||
# qwen-classifier service comment for the why and the VRAM/context math.
|
||||
# see docs/architecture.md#qwen-classifier
|
||||
LLAMA_CLASSIFIER_MODEL_FILE=Qwen3-4B-Instruct-2507-UD-Q4_K_XL.gguf
|
||||
|
||||
# --- SearXNG web search (see docs/research/litellm-searxng-search.md) ---
|
||||
# Resolved automatically by ./scripts/update.sh from search.home on this
|
||||
# host — leave blank. Only set by hand if that resolution fails (e.g.
|
||||
# search.home isn't a static DHCP reservation and its IP drifted).
|
||||
# --- SearXNG web search --- see docs/gateway.md#web-search
|
||||
# Auto-filled by update.sh — leave blank.
|
||||
SEARXNG_LAN_IP=
|
||||
|
||||
# --- OmniRoute gateway (see docs/proxy-key-onboarding.md, docs/network-access.md) ---
|
||||
# OMNIROUTE_PORT is the host-published port (reverse-proxied by NPM) — kept
|
||||
# at 4000, same as the old LiteLLM setup, so existing NPM/firewall config
|
||||
# doesn't need to change. It's mapped via plain Docker port publishing onto
|
||||
# API_PORT, omniroute's own container-internal port (left at its default,
|
||||
# not reconfigured to match). The dashboard (DASHBOARD_PORT) is never
|
||||
# published at all — see docker-compose.yml's omniroute service comment.
|
||||
# --- OmniRoute gateway --- see docs/network-access.md#gateway
|
||||
OMNIROUTE_API_PORT=20129
|
||||
OMNIROUTE_DASHBOARD_PORT=20128
|
||||
# SSE inactivity timeout before OmniRoute gives up on a streaming request and
|
||||
# cancels it (which cancels the matching llama-server task too). 180s gives
|
||||
# contended prefill (see LLAMA_PARALLEL above) room to produce a first token.
|
||||
# see docs/gateway.md#timeouts
|
||||
OMNIROUTE_STREAM_IDLE_TIMEOUT_MS=180000
|
||||
# Wait budget for the *first* SSE token specifically (distinct from the
|
||||
# inter-chunk timeout above) — see
|
||||
# docs/research/omniroute-non-ping-sse-stream-timeout.md. 30 min covers a
|
||||
# contended, large-context prefill even after retries eat into the budget.
|
||||
OMNIROUTE_REQUEST_TIMEOUT_MS=1800000
|
||||
# Random values, filled in automatically by ./scripts/update.sh — leave
|
||||
# blank. Bootstrap dashboard admin password (log in at the dashboard port,
|
||||
# change it there afterwards — this is only the first-boot value):
|
||||
# Secrets: auto-filled by update.sh — leave blank. See docs/operations.md#secrets
|
||||
OMNIROUTE_INITIAL_PASSWORD=
|
||||
# Signs dashboard session cookies:
|
||||
OMNIROUTE_JWT_SECRET=
|
||||
# Encrypts API key values at rest in omniroute's SQLite DB:
|
||||
OMNIROUTE_API_KEY_SECRET=
|
||||
# Encrypts the whole SQLite DB at rest. Do not change after first run —
|
||||
# existing encrypted data becomes unreadable if you do (same caveat as
|
||||
# LiteLLM's old LITELLM_SALT_KEY):
|
||||
# NEVER change after first run — OmniRoute's existing data becomes unreadable.
|
||||
OMNIROUTE_STORAGE_ENCRYPTION_KEY=
|
||||
# Per-deployment salts — random is fine, just needs to be stable:
|
||||
OMNIROUTE_MACHINE_ID_SALT=
|
||||
OMNIROUTE_CLI_SALT=
|
||||
# Required (production) — shared secret for the internal Codex Responses
|
||||
# WebSocket bridge. Random value, filled in automatically:
|
||||
OMNIROUTE_WS_BRIDGE_SECRET=
|
||||
# Per-workload virtual keys (one per client that calls the gateway) have no
|
||||
# scripted /key/generate equivalent yet — omniroute's key-creation endpoint
|
||||
# needs a dashboard login session, not a static bearer key (see issue #37).
|
||||
# Mint them by hand in the dashboard, add a KEY=value line here per workload
|
||||
# as you onboard one. See docs/proxy-key-onboarding.md.
|
||||
|
||||
# --- GPU access (group_add GIDs) ---
|
||||
# Shared by every GPU-touching service (llama-server, qwen-classifier)
|
||||
# for group_add: — resolved to real host GIDs by ./scripts/update.sh
|
||||
# rather than left as plain group names in docker-compose.yml, because Docker
|
||||
# resolves a *named* group_add entry against the container's own /etc/group,
|
||||
# not the host's, and fails unpredictably when the image doesn't define one
|
||||
# (worse with multiple GPU services racing on the same lookup at once — see
|
||||
# docs/research/rocm-gpu-pin-and-render-group.md and issue #5). Leave blank.
|
||||
# --- GPU access --- see docs/architecture.md#gpu-access
|
||||
# Auto-filled by update.sh — leave blank.
|
||||
HOST_VIDEO_GID=
|
||||
HOST_RENDER_GID=
|
||||
Reference in new issue
Block a user