haylan/LLM-Server
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

Why lazytainer idle-stop doesn't work today, and how the switch script should handle it #40

New Issue
Closed
opened 2026-09-05 17:36:35 +00:00 by haylan · 1 comment
haylan commented 2026-09-05 17:36:35 +00:00
Owner
Copy Link
Copy Source

Part of #38

Question

lazytainer is configured on llama-server (see docker-compose.yml labels) but
its idle-stop doesn't currently trigger — OmniRoute appears to keep pinging
it (health checks / keepalive) in a way that looks like activity to
lazytainer's packet-threshold detector, so it never sleeps. Confirm this
mechanism (check OmniRoute's provider health-check interval/behavior and
lazytainer's detection method) and recommend how scripts/switch-model.sh
(ticket, blocked by this one) should handle it: bypass lazytainer entirely
with direct docker compose stop/up for these two services, pause/stop the
OmniRoute container during the swap, reconfigure lazytainer's detection
threshold, or something else. Output: root cause + recommended approach for
the switch script.

Part of #38 ## Question lazytainer is configured on llama-server (see docker-compose.yml labels) but its idle-stop doesn't currently trigger — OmniRoute appears to keep pinging it (health checks / keepalive) in a way that looks like activity to lazytainer's packet-threshold detector, so it never sleeps. Confirm this mechanism (check OmniRoute's provider health-check interval/behavior and lazytainer's detection method) and recommend how scripts/switch-model.sh (ticket, blocked by this one) should handle it: bypass lazytainer entirely with direct docker compose stop/up for these two services, pause/stop the OmniRoute container during the swap, reconfigure lazytainer's detection threshold, or something else. Output: root cause + recommended approach for the switch script.
haylan added the wayfinder:research label 2026-09-05 17:36:35 +00:00
haylan commented 2026-09-05 18:57:36 +00:00
Author
Owner
Copy Link
Copy Source

Confirmed: lazytainer's packet-threshold detector (gopacket/libpcap BPF filter on the port, counting every packet, no source/traffic-class filtering — confirmed in vmorganp/Lazytainer source) can't distinguish OmniRoute's periodic provider health-check pings (default every <=5min via CREDENTIAL_HEALTH_CHECK_INTERVAL, global on/off only, no per-provider exclude) from real inference traffic on the same port 8080 — no threshold tuning fixes this, it's a ceiling in the tool. Recommend scripts/switch-model.sh (#43) bypass lazytainer entirely and drive docker compose stop/up -d directly on llama-server/comfyui for the swap, rather than relying on idle-stop. Full writeup: docs/research/lazytainer-omniroute-idle-stop.md

Confirmed: lazytainer's packet-threshold detector (gopacket/libpcap BPF filter on the port, counting every packet, no source/traffic-class filtering — confirmed in vmorganp/Lazytainer source) can't distinguish OmniRoute's periodic provider health-check pings (default every <=5min via CREDENTIAL_HEALTH_CHECK_INTERVAL, global on/off only, no per-provider exclude) from real inference traffic on the same port 8080 — no threshold tuning fixes this, it's a ceiling in the tool. Recommend scripts/switch-model.sh (#43) bypass lazytainer entirely and drive docker compose stop/up -d directly on llama-server/comfyui for the swap, rather than relying on idle-stop. Full writeup: docs/research/lazytainer-omniroute-idle-stop.md
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
wayfinder:grilling
Wayfinder grilling ticket
 wayfinder:map
Wayfinder map issue
 wayfinder:prototype
Wayfinder prototype ticket
 wayfinder:research
Wayfinder research ticket
 wayfinder:task
Wayfinder task ticket
No labels wayfinder:research
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
haylan (Arthur Erlich)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Blocks
#43 scripts/switch-model.sh: swap GPU residency between Qwen and ComfyUI
haylan/LLM-Server
Reference: haylan/LLM-Server#40