OmniRoute: docker-compose service design and secrets/config plan #37

New Issue

2026-09-03T17:38:36Z

haylan commented

2026-09-03 17:38:36 +00:00

Part of #31

Question

Turn #32/#35/#36's findings into a concrete docker-compose.yml service (replacing litellm) and config/secrets plan, resolving the map's "Not yet specified" design item. User has explicitly overridden the standing recommendation against integrating OmniRoute (2026-09-04) and directed implementation to proceed.

Answer

Image & profile: diegosouzapw/omniroute:latest (pin an exact version later, e.g. :3.8.51, once smoke-tested — :latest for initial dev). Multi-arch (AMD64+ARM64), matches this host.

Networking — LAN-only dashboard, internet-facing API: split-port mode. API_HOST=0.0.0.0, API_PORT=20129, DASHBOARD_PORT=20128. Only API_PORT is published in ports: (reverse-proxied by NPM to proxy.ai.home/proxy.ai.haylan.ch, same pattern as today's litellm:4000). DASHBOARD_PORT is never published — no NPM path-deny rule needed, tighter than LiteLLM's current /ui workaround.

Local provider registration (per #32/#35, done once via the dashboard or POST /api/providers, not a static config file):

Chat: llama-cpp provider preset → baseUrl: http://llama-server:8080/v1
Embeddings: openai-compatible-embeddings provider_node → baseUrl: http://embedding-server:8080/v1
Search: searxng-search provider → baseUrl: http://search.home/ (needs extra_hosts for search.home, same as today, plus OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS=true / OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS=true set, since LAN/private provider URLs are gated by default)

Secrets (scripts/update.sh generates on first run, replacing LITELLM_MASTER_KEY/LITELLM_SALT_KEY):

INITIAL_PASSWORD — bootstrap dashboard admin password
JWT_SECRET — openssl rand -base64 48
API_KEY_SECRET — openssl rand -hex 32
STORAGE_ENCRYPTION_KEY — openssl rand -hex 32
MACHINE_ID_SALT, OMNIROUTE_CLI_SALT — per-deployment salts, random

Persistence: named volume omniroute-data:/app/data (or DATA_DIR override) — required per OmniRoute's own docs ("always mount a volume... to persist your database, keys, and configurations"). Bundled Redis (omniroute-redis-data volume) is always included, no profile gate — can likely reuse this stack's existing redis service instead via REDIS_* env vars rather than running a second Redis; needs a quick check against OmniRoute's Redis env var names at implementation time.

Kept unchanged (per #34): litellm-pgvector, pgvector-db, qdrant — OmniRoute has no memory/RAG parity, this stack's knowledgebase stays on the existing companion service, just re-pointed at OmniRoute for its own embedding calls if litellm-pgvector's EMBEDDING__BASE_URL needs to change.

Per-workload keys: POST /api/keys replaces LiteLLM's /key/generate call pattern in scripts/update.sh's mint_key_if_blank — exact request/response shape needs a live check against the deployed instance (scope model was underdocumented per #36).

Part of #31 ## Question Turn #32/#35/#36's findings into a concrete `docker-compose.yml` service (replacing `litellm`) and config/secrets plan, resolving the map's "Not yet specified" design item. User has explicitly overridden the standing recommendation against integrating OmniRoute (2026-09-04) and directed implementation to proceed. ## Answer **Image & profile**: `diegosouzapw/omniroute:latest` (pin an exact version later, e.g. `:3.8.51`, once smoke-tested — `:latest` for initial dev). Multi-arch (AMD64+ARM64), matches this host. **Networking — LAN-only dashboard, internet-facing API**: split-port mode. `API_HOST=0.0.0.0`, `API_PORT=20129`, `DASHBOARD_PORT=20128`. Only `API_PORT` is published in `ports:` (reverse-proxied by NPM to `proxy.ai.home`/`proxy.ai.haylan.ch`, same pattern as today's `litellm:4000`). `DASHBOARD_PORT` is never published — no NPM path-deny rule needed, tighter than LiteLLM's current `/ui` workaround. **Local provider registration** (per #32/#35, done once via the dashboard or `POST /api/providers`, not a static config file): - Chat: `llama-cpp` provider preset → `baseUrl: http://llama-server:8080/v1` - Embeddings: `openai-compatible-embeddings` provider_node → `baseUrl: http://embedding-server:8080/v1` - Search: `searxng-search` provider → `baseUrl: http://search.home/` (needs `extra_hosts` for `search.home`, same as today, plus `OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS=true` / `OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS=true` set, since LAN/private provider URLs are gated by default) **Secrets** (`scripts/update.sh` generates on first run, replacing `LITELLM_MASTER_KEY`/`LITELLM_SALT_KEY`): - `INITIAL_PASSWORD` — bootstrap dashboard admin password - `JWT_SECRET` — `openssl rand -base64 48` - `API_KEY_SECRET` — `openssl rand -hex 32` - `STORAGE_ENCRYPTION_KEY` — `openssl rand -hex 32` - `MACHINE_ID_SALT`, `OMNIROUTE_CLI_SALT` — per-deployment salts, random **Persistence**: named volume `omniroute-data:/app/data` (or `DATA_DIR` override) — required per OmniRoute's own docs ("always mount a volume... to persist your database, keys, and configurations"). Bundled Redis (`omniroute-redis-data` volume) is always included, no profile gate — can likely reuse this stack's existing `redis` service instead via `REDIS_*` env vars rather than running a second Redis; needs a quick check against OmniRoute's Redis env var names at implementation time. **Kept unchanged** (per #34): `litellm-pgvector`, `pgvector-db`, `qdrant` — OmniRoute has no memory/RAG parity, this stack's knowledgebase stays on the existing companion service, just re-pointed at OmniRoute for its own embedding calls if `litellm-pgvector`'s `EMBEDDING__BASE_URL` needs to change. **Per-workload keys**: `POST /api/keys` replaces LiteLLM's `/key/generate` call pattern in `scripts/update.sh`'s `mint_key_if_blank` — exact request/response shape needs a live check against the deployed instance (scope model was underdocumented per #36).

haylan added the wayfinder:task label 2026-09-03 17:38:36 +00:00

haylan self-assigned this 2026-09-03 17:38:44 +00:00

haylan commented

2026-09-03 17:38:45 +00:00

Resolved — design captured in the issue body above. Proceeding to implementation on main (docker-compose.yml, .env.example, scripts/update.sh).

haylan closed this issue

2026-09-03 17:38:46 +00:00

haylan commented

2026-09-03 17:49:53 +00:00

Implemented on main (commit 472e3a4): docker-compose.yml, .env.example, scripts/update.sh, and docs updated per the design above. Also, separately, the user asked to drop the gateway-level memory/knowledgebase feature entirely (litellm-pgvector, pgvector-db, embedding-server) — unrelated to OmniRoute's lack of parity found in #34, just no longer wanted. Compose config validated (docker compose config -q) and update.sh syntax-checked; not yet run against the live R9700 server.

Implemented on main (commit 472e3a4): docker-compose.yml, .env.example, scripts/update.sh, and docs updated per the design above. Also, separately, the user asked to drop the gateway-level memory/knowledgebase feature entirely (litellm-pgvector, pgvector-db, embedding-server) — unrelated to OmniRoute's lack of parity found in #34, just no longer wanted. Compose config validated (docker compose config -q) and update.sh syntax-checked; not yet run against the live R9700 server.

haylan commented

2026-09-03 17:55:56 +00:00

Follow-up (commit 3bbda09): Open WebUI and Qdrant removed from the stack entirely per user request — no longer needed, every client is now a coding CLI. This drops the open-webui service, its OPENWEBUI_OMNIROUTE_KEY wiring, and qdrant (which only ever served Open WebUI's own memory/RAG). docker-compose.yml's omniroute service and everything else from this ticket's design is unaffected.

Follow-up (commit 3bbda09): Open WebUI and Qdrant removed from the stack entirely per user request — no longer needed, every client is now a coding CLI. This drops the open-webui service, its OPENWEBUI_OMNIROUTE_KEY wiring, and qdrant (which only ever served Open WebUI's own memory/RAG). docker-compose.yml's omniroute service and everything else from this ticket's design is unaffected.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: haylan/LLM-Server#37