feat(omniroute): add dedicated embedding-server for memory feature

llama.cpp loads one model per process and the running Qwen3.8-27B chat
model isn't embedding-trained, so this is a second, CPU-only
llama-server instance (bge-small-en-v1.5, 384-dim) rather than adding
--embeddings to the chat one — see docs/research/litellm-knowledgebase.md
#3.

Downloader extended to fetch both GGUFs into the shared models volume.
No host port published — OmniRoute reaches it via the ai-stack network
DNS name (embedding-server:8081).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qx22CV9EUS3hGATucQav13
This commit is contained in:
2026-09-03 21:43:20 +02:00
co-authored by Claude-Bot
parent 885de477ba
commit 9e1362c22c
3 changed files with 36 additions and 3 deletions
+6
View File
@@ -28,6 +28,12 @@ LLAMA_GPU_LAYERS=999
# of dropping context: --cache-type-k/v q8_0 roughly halves it.
LLAMA_CTX_SIZE=131072
# --- embedding-server (OmniRoute memory) ---
# bge-small-en-v1.5, CPU-only, 384-dim output — see docker-compose.yml's
# embedding-server service comment for why this is a second model/process
# rather than reusing llama-server.
EMBEDDING_MODEL_FILE=bge-small-en-v1.5-q8_0.gguf
# --- Lazytainer ---
# Seconds of inactivity before llama-server is stopped. 900 = 15 min.
LAZYTAINER_INACTIVE_TIMEOUT=900