haylan/LLM-Server
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

fix: GPU pinned at 100% with two llama.cpp containers, flaky render group

Browse Source 
Two real-hardware findings from issue #5, both researched and fixed
together (docs/research/rocm-gpu-pin-and-render-group.md):

- GPU_MAX_HW_QUEUES=1 on llama-server and llama-server-fast. Confirmed
  on real hardware: either container alone is fine (3% GPU, low
  power), only two concurrent HIP contexts pin the R9700 at 100%/
  boost-clock (ROCm/ROCm#5706, an MES firmware bug). The env var is
  validated on the exact image this stack uses, per-process by design
  — applying it to both containers is the correct scope.

- group_add switched from plain names (video/render) to resolved
  numeric GIDs (HOST_VIDEO_GID/HOST_RENDER_GID) on all three GPU
  services. The "unable to find group render: no matching entries in
  group file" error confirmed new since the second GPU service was
  added is a known Docker bug (docker/cli#4714): group_add by name
  resolves against the container's own /etc/group, not the host's,
  and multiple GPU services starting concurrently race on that lookup.
  Numeric GIDs skip resolution entirely. scripts/update.sh's existing
  comfyui-only GID resolution is generalized to resolve these once for
  all three services.

docker compose config -q validated (fails fast with a clear error if
the GIDs aren't resolved yet, passes once they are).

Refs #5

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MrnMEdzeQzqZE5soVEXPCx
This commit is contained in:
haylan
2026-09-06 21:33:40 +02:00
co-authored by Claude-Bot
parent 1ee2e76033
commit 75033dacd7
4 changed files with 142 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/update.sh
+12 -5
View File
@@ -56,22 +56,29 @@ else
echo "SEARXNG_LAN_IP: couldn't resolve search.home from this host, set it manually if still blank."
fi
echo "==> resolving ComfyUI host GID/UID"
echo "==> resolving ComfyUI host UID"
# yurisasc/comfyui-rocm7.1 wants these as env vars, not just group_add in
# compose — resolve from this host, same pattern as SEARXNG_LAN_IP.
set_if_blank COMFYUI_PUID "$(id -u)"
set_if_blank COMFYUI_PGID "$(id -g)"
echo "==> resolving host video/render GIDs (shared by every GPU service)"
# Numeric GIDs, not names, in docker-compose.yml's group_add: — Docker
# resolves a *named* group_add entry against the container's own /etc/group,
# not the host's, and fails unpredictably (worse with multiple GPU services
# starting concurrently and racing on the same lookup) — see
# docs/research/rocm-gpu-pin-and-render-group.md and issue #5.
video_gid="$(getent group video 2>/dev/null | cut -d: -f3)"
render_gid="$(getent group render 2>/dev/null | cut -d: -f3)"
if [ -n "$video_gid" ]; then
set_if_blank COMFYUI_VIDEO_GID "$video_gid"
set_if_blank HOST_VIDEO_GID "$video_gid"
else
echo "COMFYUI_VIDEO_GID: no 'video' group on this host, set it manually if still blank."
echo "HOST_VIDEO_GID: no 'video' group on this host, set it manually if still blank."
fi
if [ -n "$render_gid" ]; then
set_if_blank COMFYUI_RENDER_GID "$render_gid"
set_if_blank HOST_RENDER_GID "$render_gid"
else
echo "COMFYUI_RENDER_GID: no 'render' group on this host, set it manually if still blank."
echo "HOST_RENDER_GID: no 'render' group on this host, set it manually if still blank."
fi
echo "==> git pull"