haylan/LLM-Server
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

Local image generation via ComfyUI + OmniRoute #38

New Issue
Open
opened 2026-09-05 17:36:13 +00:00 by haylan · 0 comments
haylan commented 2026-09-05 17:36:13 +00:00
Owner
Copy Link
Copy Source

Destination

ComfyUI running as a local image-gen backend on the LLM-Server box, registered
with OmniRoute as its ComfyUI/SD-WebUI-local provider, reachable via CLI
through an OmniRoute API key (same pattern as Qwen today). A manual switch
script swaps GPU residency between Qwen and ComfyUI — they never run
concurrently, VRAM doesn't fit both. The downloader service is extended to
fetch the diffusion model weights. ComfyUI is also published on host port
8138 for a future external nginx route to comfy.home (outside this stack).

Notes

  • Domain: docs/agents/domain.md, CONTEXT.md, docs/adr/.
  • Related research: docs/research/image-generation-options.md (backend/model
    survey done before this map — ComfyUI was initially ruled out there in
    favor of exploring Forge, then reversed after checking ComfyUI's repo
    directly).
  • GPU: AMD R9700, 32GB VRAM, ROCm/gfx1201 — not CUDA.
  • Current VRAM baseline: llama-server (Qwen3.8-27B) uses ~25.6GB per
    .env.example's own math (real measured closer to ~75%), leaving ~6GB
    free while it's running.
  • Model choice is deliberately left open pending research (#39): a bigger
    model than FLUX.1-schnell may be viable once Qwen is fully stopped and the
    full ~32GB is free.
  • Known related bug to account for (#40): lazytainer's idle-stop doesn't
    currently work — OmniRoute's own health-check pings keep llama-server
    looking "active", defeating the inactivity timer. The switch script may
    need to bypass lazytainer entirely (direct docker compose stop/up)
    and/or pause OmniRoute during the swap.

Decisions so far

  • Local image generation via ComfyUI + OmniRoute — map created; destination scoped to ComfyUI (not Forge/ComfyUI-alternative), model choice deferred to research, dashboard/chat-UI enablement out of scope for now.
  • Why lazytainer idle-stop doesn't work today, and how the switch script should handle it — lazytainer's packet-threshold detector is source-blind and can't distinguish OmniRoute's periodic health-check pings from real traffic on the same port, so no threshold tuning fixes it; scripts/switch-model.sh (#43) should bypass lazytainer entirely and drive docker compose stop/up -d directly for the swap.
  • Best diffusion model given full ~32GB headroom (Qwen stopped) — move up from FLUX.1-schnell to Qwen-Image at FP8 (~25GB): Apache-2.0 license, and the only candidate with a ComfyUI workflow pre-validated specifically on this GPU architecture (gfx1201/R9700); HunyuanImage-3.0 ruled out (CUDA-only, multi-GPU VRAM floor), Krea-2 too new for R9700-specific evidence yet.

Not yet specified

  • OmniRoute dashboard/chat-UI enablement — flagged as a nice-to-have, not
    required for this destination; no ticket unless it becomes blocking.

Out of scope

  • Concurrent GPU residency (Qwen + ComfyUI loaded at once) — VRAM doesn't
    fit both; switching is always exclusive.

Tickets

## Destination ComfyUI running as a local image-gen backend on the LLM-Server box, registered with OmniRoute as its ComfyUI/SD-WebUI-local provider, reachable via CLI through an OmniRoute API key (same pattern as Qwen today). A manual switch script swaps GPU residency between Qwen and ComfyUI — they never run concurrently, VRAM doesn't fit both. The downloader service is extended to fetch the diffusion model weights. ComfyUI is also published on host port 8138 for a future external nginx route to `comfy.home` (outside this stack). ## Notes - Domain: `docs/agents/domain.md`, `CONTEXT.md`, `docs/adr/`. - Related research: `docs/research/image-generation-options.md` (backend/model survey done before this map — ComfyUI was initially ruled out there in favor of exploring Forge, then reversed after checking ComfyUI's repo directly). - GPU: AMD R9700, 32GB VRAM, ROCm/gfx1201 — not CUDA. - Current VRAM baseline: llama-server (Qwen3.8-27B) uses ~25.6GB per `.env.example`'s own math (real measured closer to ~75%), leaving ~6GB free while it's running. - Model choice is deliberately left open pending research (#39): a bigger model than FLUX.1-schnell may be viable once Qwen is fully stopped and the full ~32GB is free. - Known related bug to account for (#40): lazytainer's idle-stop doesn't currently work — OmniRoute's own health-check pings keep llama-server looking "active", defeating the inactivity timer. The switch script may need to bypass lazytainer entirely (direct `docker compose stop/up`) and/or pause OmniRoute during the swap. ## Decisions so far - [Local image generation via ComfyUI + OmniRoute](https://git.arthurerlich.de/haylan/LLM-Server/issues/38) — map created; destination scoped to ComfyUI (not Forge/ComfyUI-alternative), model choice deferred to research, dashboard/chat-UI enablement out of scope for now. - [Why lazytainer idle-stop doesn't work today, and how the switch script should handle it](https://git.arthurerlich.de/haylan/LLM-Server/issues/40) — lazytainer's packet-threshold detector is source-blind and can't distinguish OmniRoute's periodic health-check pings from real traffic on the same port, so no threshold tuning fixes it; `scripts/switch-model.sh` (#43) should bypass lazytainer entirely and drive `docker compose stop`/`up -d` directly for the swap. - [Best diffusion model given full ~32GB headroom (Qwen stopped)](https://git.arthurerlich.de/haylan/LLM-Server/issues/39) — move up from FLUX.1-schnell to **Qwen-Image at FP8** (~25GB): Apache-2.0 license, and the only candidate with a ComfyUI workflow pre-validated specifically on this GPU architecture (gfx1201/R9700); HunyuanImage-3.0 ruled out (CUDA-only, multi-GPU VRAM floor), Krea-2 too new for R9700-specific evidence yet. ## Not yet specified - OmniRoute dashboard/chat-UI enablement — flagged as a nice-to-have, not required for this destination; no ticket unless it becomes blocking. ## Out of scope - Concurrent GPU residency (Qwen + ComfyUI loaded at once) — VRAM doesn't fit both; switching is always exclusive. ## Tickets - [ ] Add comfyui service to docker-compose.yml, register with OmniRoute (#41) - [ ] Extend downloader for the diffusion model weights (#42) — blocked by #39 - [ ] scripts/switch-model.sh: swap GPU residency between Qwen and ComfyUI (#43) — blocked by #40
haylan added the wayfinder:map label 2026-09-05 17:36:13 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
wayfinder:grilling
Wayfinder grilling ticket
 wayfinder:map
Wayfinder map issue
 wayfinder:prototype
Wayfinder prototype ticket
 wayfinder:research
Wayfinder research ticket
 wayfinder:task
Wayfinder task ticket
No labels wayfinder:map
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
haylan (Arthur Erlich)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: haylan/LLM-Server#38