Arthur Erlich

haylan pushed to main at haylan/LLM-Server

2026-08-25 05:21:01 +00:00

487573c533 Add build plan documentation for 4x Radeon AI PRO R9700 rackmount LLM server

haylan commented on issue haylan/LLM-Server#15

2026-08-25 05:18:10 +00:00

Migrate Open WebUI (and existing consumers) to route through the new proxy

Resolved: migrated Open WebUI and the coding CLIs to the proxy.

Open WebUI's OPENAI_API_BASE_URL now points at litellm:4000 instead of llama-server:8080, using a provisioned virtual key…

haylan pushed to main at haylan/LLM-Server

2026-08-25 05:18:04 +00:00

5cb34b19f3 Migrate Open WebUI and coding CLIs to the AI proxy (resolves #15)

haylan closed issue haylan/LLM-Server#15

2026-08-25 05:18:04 +00:00

Migrate Open WebUI (and existing consumers) to route through the new proxy

haylan commented on issue haylan/LLM-Server#14

2026-08-25 05:11:24 +00:00

Author the docker-compose service for the chosen proxy

Resolved: authored the docker-compose service.

Added litellm + litellm-db (Postgres) services to docker-compose.yml, a new litellm-config.yaml wiring the local llama.cpp model with #11's…

haylan pushed to main at haylan/LLM-Server

2026-08-25 05:11:17 +00:00

0aefb36a48 Author LiteLLM AI proxy service (resolves #14)

haylan closed issue haylan/LLM-Server#14

2026-08-25 05:11:17 +00:00

Author the docker-compose service for the chosen proxy

haylan opened issue haylan/LLM-Server#17

2026-08-25 05:11:03 +00:00

Verify the AI proxy stack on the real R9700 box (LiteLLM scheduler smoke test)

haylan commented on issue haylan/LLM-Server#16

2026-08-25 05:07:28 +00:00

How should the proxy queue and prioritize concurrent requests across workloads?

Resolved via grilling.

Two tiers, assigned per workload's virtual key: high priority (Open WebUI chat, coding CLIs, Gitea code review — interactive) vs. low priority (Paperless OCR/tagging,…

haylan pushed to main at haylan/LLM-Server

2026-08-25 05:07:21 +00:00

0342090e1c Document AI proxy request priority/queueing plan (resolves #16)

haylan closed issue haylan/LLM-Server#16

2026-08-25 05:07:21 +00:00

How should the proxy queue and prioritize concurrent requests across workloads?

haylan commented on issue haylan/LLM-Server#13

2026-08-25 05:05:08 +00:00

Network/hostname plan for exposing the proxy (LAN + external)

Resolved via grilling.

Hostnames: proxy.ai.home (LAN) / proxy.ai.haylan.ch (external) — new NPM Proxy Hosts, same pattern as Open WebUI's ai.home/ai.haylan.ch. LiteLLM's own virtual-key…

haylan pushed to main at haylan/LLM-Server

2026-08-25 05:05:02 +00:00

7e50e1867f Document AI proxy network exposure plan (resolves #13)

haylan closed issue haylan/LLM-Server#13

2026-08-25 05:05:02 +00:00

Network/hostname plan for exposing the proxy (LAN + external)

haylan commented on issue haylan/LLM-Server#12

2026-08-25 04:57:10 +00:00

How should per-workload API keys/accounts be provisioned, rotated, and documented?

Resolved via grilling.

Keys created by hand in LiteLLM's Admin UI, named as a workload-purpose slug (e.g. paperless-ocr, gitea-code-review) — the name is the ledger, no separate tracking…

haylan pushed to main at haylan/LLM-Server

2026-08-25 04:56:53 +00:00

1c29e07762 Document AI proxy key onboarding process (resolves #12)

haylan closed issue haylan/LLM-Server#12

2026-08-25 04:56:53 +00:00

How should per-workload API keys/accounts be provisioned, rotated, and documented?

haylan commented on issue haylan/LLM-Server#11

2026-08-25 04:50:14 +00:00

Reference cloud model/pricing for the shadow-cost estimate

Recommendation: Claude Sonnet 5, fixed single reference. $2/MTok input, $10/MTok output (current standard pricing per Anthropic's pricing page, checked 2026-08-25) — matches what the user…

haylan closed issue haylan/LLM-Server#11

2026-08-25 04:49:50 +00:00

Reference cloud model/pricing for the shadow-cost estimate

haylan pushed to research/proxy-shadow-pricing at haylan/LLM-Server

2026-08-25 04:49:40 +00:00

a751e656e3 research: reference cloud pricing for LiteLLM shadow-cost estimate