haylanandClaude Bot
408ed3e586
Log shows 253 prompt-cache evictions and 146 full ~42k re-prefills at the default budget. Add LLAMA_CACHE_RAM and the throughput-tuning research note. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
37 lines
1.2 KiB
Bash
37 lines
1.2 KiB
Bash
# Template for .env, managed by ./scripts/update.sh — see docs/operations.md#env-sync
|
|
|
|
# --- llama.cpp / models --- see docs/architecture.md#llama-server
|
|
LLAMA_MODEL_FILE=Qwen3.8-27B-UD-Q4_K_XL.gguf
|
|
LLAMA_GPU_LAYERS=999
|
|
LLAMA_CTX_SIZE=262144
|
|
LLAMA_PARALLEL=2
|
|
LLAMA_CACHE_RAM=24576
|
|
|
|
LLAMA_DRAFT_MODEL_FILE=Qwen3.8-27B-DFlash2-Q4_K_M.gguf
|
|
LLAMA_SPEC_N_MAX=7
|
|
|
|
# --- SearXNG web search --- see docs/gateway.md#web-search
|
|
# Auto-filled by update.sh — leave blank.
|
|
SEARXNG_LAN_IP=
|
|
|
|
# --- OmniRoute gateway --- see docs/network-access.md#gateway
|
|
OMNIROUTE_API_PORT=20129
|
|
OMNIROUTE_DASHBOARD_PORT=20128
|
|
# see docs/gateway.md#timeouts
|
|
OMNIROUTE_STREAM_IDLE_TIMEOUT_MS=180000
|
|
OMNIROUTE_REQUEST_TIMEOUT_MS=1800000
|
|
# Secrets: auto-filled by update.sh — leave blank. See docs/operations.md#secrets
|
|
OMNIROUTE_INITIAL_PASSWORD=
|
|
OMNIROUTE_JWT_SECRET=
|
|
OMNIROUTE_API_KEY_SECRET=
|
|
# NEVER change after first run — OmniRoute's existing data becomes unreadable.
|
|
OMNIROUTE_STORAGE_ENCRYPTION_KEY=
|
|
OMNIROUTE_MACHINE_ID_SALT=
|
|
OMNIROUTE_CLI_SALT=
|
|
OMNIROUTE_WS_BRIDGE_SECRET=
|
|
|
|
# --- GPU access --- see docs/architecture.md#gpu-access
|
|
# Auto-filled by update.sh — leave blank.
|
|
HOST_VIDEO_GID=
|
|
HOST_RENDER_GID=
|