# Research: Why lazytainer's idle-stop on llama-server doesn't fire, and what switch-model.sh should do about it

**Question:** lazytainer is configured on `llama-server` (`docker-compose.yml`
`lazytainer.group.llamaserver.*` labels) but its idle-stop never triggers in
practice — OmniRoute appears to keep the container looking "active" to
lazytainer's packet-threshold detector. Confirm the mechanism, find root
cause, and recommend how the future `scripts/switch-model.sh` (#43, blocked)
should handle GPU-residency swaps between `llama-server` and a new `comfyui`
service given this.

**Answer:** Confirmed. lazytainer's detector is a dumb per-port packet
counter with no traffic classification — it cannot tell OmniRoute's
background provider health-check pings apart from real inference traffic,
and there is no config knob in lazytainer or a per-provider one in OmniRoute
that fixes this. **`switch-model.sh` should bypass lazytainer entirely** for
the swap: drive `docker compose stop`/`up -d` directly on both services,
rather than trying to make lazytainer's idle-stop cooperate.

## Current config (`docker-compose.yml`)

```yaml
labels:
  - "lazytainer.group.llamaserver.sleepMethod=stop"
  - "lazytainer.group.llamaserver.ports=8080"
  - "lazytainer.group.llamaserver.inactiveTimeout=${LAZYTAINER_INACTIVE_TIMEOUT:-900}"
  - "lazytainer.group.llamaserver.minPacketThreshold=2"
```

`ports=8080` matches the container's real internal port (`expose: ["8080"]`,
confirmed in the same file) — not a misconfiguration. `minPacketThreshold=2`
is already far *below* lazytainer's own documented default of `30`, i.e. this
deployment already tried loosening the threshold to make idle-stop easier to
reach, not harder.

## How lazytainer's detector actually works (primary source: `vmorganp/Lazytainer`)

Confirmed against the project's README and Go source
(`src/group.go`) on [github.com/vmorganp/Lazytainer](https://github.com/vmorganp/Lazytainer):

- It captures packets with **gopacket/libpcap directly on the configured
  `netInterface`** (default `eth0`), applying a BPF filter built from the
  group's `ports` list (`"port 8080"` here, per the source's filter-string
  construction, e.g. `"port 80 or port 81 or etc."` in the general case).
- The filter matches **every packet to or from the port** — SYN, ACK,
  data, FIN, everything. It is not restricted to new-connection SYNs.
- Every `pollRate` seconds (default `30`; not overridden in this repo's
  config) it samples a rolling packet counter (`rxHistory`) and compares the
  delta against `minPacketThreshold`:
  `rxHistory[0]+minPacketThreshold > rxHistory[len(rxHistory)-1]` → treated as
  active, `inactiveSeconds` resets to 0.
- `ignoreActiveClients` (default `false`, not set here) only changes whether
  an ESTABLISHED-connection count is also checked; it does not add any
  content- or source-based filtering.
- **There is no mechanism anywhere in lazytainer to exclude specific traffic
  (by source IP, path, header, or request type) from the packet count.** The
  README's config table (`ports`, `inactiveTimeout`, `minPacketThreshold`,
  `ignoreActiveClients`, `pollRate`, `sleepMethod`, `netInterface`) is
  exhaustive — nothing else exists to tune this per-caller.

Consequence: a single TCP connection to port 8080 — a bare connect + one
small HTTP exchange + close — already produces well over `minPacketThreshold=2`
packets purely from the handshake and teardown (SYN, SYN-ACK, ACK, ..., FIN,
ACK), regardless of payload size or purpose. At this threshold, essentially
*any* connection to the port counts as "active" and resets `inactiveTimeout`.
Raising the threshold wouldn't help either — the fix would need to be
"ignore packets from OmniRoute's health-checker," which the tool has no way
to express; it only counts packets on a port, source-blind.

## How OmniRoute actually touches registered providers (primary source: `diegosouzapw/OmniRoute`)

Confirmed against
[`docs/reference/ENVIRONMENT.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/reference/ENVIRONMENT.md)
in the OmniRoute repo:

- OmniRoute runs a **background credential/connection health-check
  scheduler** (`src/lib/credentialHealth/scheduler.ts`) on
  `CREDENTIAL_HEALTH_CHECK_INTERVAL`, default `300000` ms (5 min), minimum
  `10000` ms (10s) — this periodically re-tests each registered provider's
  connection, which for a provider like `llama-server` (a plain HTTP base
  URL, no API key) means an actual request/connection to
  `llama-server:8080`.
- Results are cached for `CREDENTIAL_HEALTH_CACHE_TTL` (default also 5 min).
- **Only one exclusion exists, and it's hardcoded by provider category, not
  configurable per-provider**: search providers
  (`SEARCH_VALIDATOR_CONFIGS` in
  `src/lib/providers/validation/searchProviders.ts`, e.g. `tavily-search`)
  are permanently skipped because their validation call is a real billed
  upstream query. `llama-server` is an inference provider, not a search
  provider — it is not in this exclusion list.
- The only toggle that actually stops the sweep is global:
  `OMNIROUTE_DISABLE_CREDENTIAL_HEALTH_CHECK=1`/`true`, which "disable[s]
  background periodic testing of provider connections" for **every**
  registered provider at once. There is no documented per-provider
  disable/pause flag in
  [`docs/reference/PROVIDER_REFERENCE.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/reference/PROVIDER_REFERENCE.md) —
  the dashboard's `/dashboard/providers` page is described only as where you
  "enable, configure, and test each provider," with no documented
  independent "pause health checks for this one provider" control.

So: OmniRoute is not the sole cause, but it is a live, recurring cause. Every
5 minutes (at most — could also be triggered ad hoc by dashboard/API use) it
opens a connection to `llama-server:8080` purely to check the provider is
alive, which is exactly the kind of traffic lazytainer's port-level counter
cannot distinguish from real inference calls. With `inactiveTimeout=900`
(15 min) and a health-check every ≤300s, the container practically always
sees qualifying traffic before its idle timer would expire.

## Root cause

Two independent, both-true facts combine to defeat idle-stop:

1. **lazytainer's detector is fundamentally traffic-blind** — it counts raw
   packets on a port with no way to exclude any specific caller or traffic
   class. This is a property of the tool, not a misconfiguration in this
   repo (`ports=8080` is correct; `minPacketThreshold=2` is already at the
   permissive end).
2. **OmniRoute periodically pings every registered non-search provider**
   (default every ≤5 min) to keep its health/availability status current,
   and that ping is indistinguishable, at the packet level, from a real
   inference request.

Neither side offers a targeted fix: lazytainer has no allowlist/denylist by
source, and OmniRoute's only "stop pinging" lever
(`OMNIROUTE_DISABLE_CREDENTIAL_HEALTH_CHECK`) is all-or-nothing across every
provider, not scoped to just `llama-server`. Tuning `minPacketThreshold`
higher or lower doesn't change the outcome either way, since the health-check
traffic and real traffic land on the exact same port with no distinguishing
packet-level signature.

## Recommendation for `scripts/switch-model.sh` (#43)

**Bypass lazytainer entirely for the GPU-residency swap.** Drive both
services directly:

```bash
docker compose stop llama-server
docker compose up -d comfyui
# ...and the reverse when swapping back
```

Justification:

- The swap is a **deliberate, scripted, known-in-advance** event — the
  script always knows exactly which service should go up and which should
  go down. Idle-stop detection exists to handle the case where nobody knows
  when a service last had traffic; that's not this case, so routing the
  swap through a passive heuristic (lazytainer's idle timer) that this
  research shows is already unreliable for `llama-server` adds a point of
  failure for no benefit. Direct `docker compose stop`/`up -d` is
  deterministic and immune to the packet-counting confound described above.
- Reconfiguring lazytainer's thresholds was considered and rejected: no
  threshold value fixes a detector that cannot distinguish OmniRoute's
  keepalive traffic from real traffic on the same port (see Root cause).
  This is a ceiling in the tool itself, not a tuning problem.
- Pausing OmniRoute's polling for the swap window was also considered.
  It's the one lever available (`OMNIROUTE_DISABLE_CREDENTIAL_HEALTH_CHECK`),
  but it is global — it would blind OmniRoute's health status for *every*
  provider (including `searxng-search`, if registered) for the duration of
  the swap, and adds an extra env-toggle-and-restart step to the script for
  a problem that direct compose control sidesteps completely. It's worth
  flagging for #43's implementation as a *secondary* safety measure — briefly
  disabling the sweep (or accepting that OmniRoute may show `llama-server` as
  errored/offline for up to `CREDENTIAL_HEALTH_CHECK_INTERVAL` after it's
  stopped) — but it should not be the primary mechanism the swap relies on.
- This does **not** require removing the existing `lazytainer.group.llamaserver.*`
  labels — they can stay for whatever idle-stop benefit they still provide
  between swaps (e.g. genuinely idle periods where nothing, including
  OmniRoute, has recently touched the container long enough to matter) while
  `switch-model.sh` simply never depends on lazytainer to do the actual
  stop/start for a swap.

## Bottom line for #43 (blocked ticket, once unblocked)

- `switch-model.sh` should call `docker compose stop <from-service>` /
  `docker compose up -d <to-service>` directly — never rely on lazytainer's
  idle-stop to free the GPU as part of a swap.
- No lazytainer config change (threshold, ports, poll rate) is a viable fix;
  the detector has no way to exclude OmniRoute's traffic by source.
- Optionally, as a secondary hygiene step, the script may toggle
  `OMNIROUTE_DISABLE_CREDENTIAL_HEALTH_CHECK` around the swap (or simply
  tolerate a stale "errored" status in OmniRoute's dashboard for up to one
  `CREDENTIAL_HEALTH_CHECK_INTERVAL`) to avoid OmniRoute flagging the
  just-stopped provider as failed mid-swap — but this is cosmetic/status
  hygiene, not what makes the swap itself work.

Sources: [`vmorganp/Lazytainer`](https://github.com/vmorganp/Lazytainer)
(README config table; `src/group.go` packet-capture and threshold-comparison
logic), [`diegosouzapw/OmniRoute` —
`docs/reference/ENVIRONMENT.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/reference/ENVIRONMENT.md)
(credential health-check scheduler env vars), [`diegosouzapw/OmniRoute` —
`docs/reference/PROVIDER_REFERENCE.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/reference/PROVIDER_REFERENCE.md)
(provider dashboard controls), this repo's `docker-compose.yml`
(`lazytainer.group.llamaserver.*` labels, `llama-server`/`omniroute` service
definitions).