haylan/LLM-Server
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

Commit Graph

  • 71fb83f760 Merge pull request 'docs(research): add qwen3.8-27b decode speed analysis' (#68) from docs/decode-speed-research into main main haylan 2026-11-11 15:52:21 +00:00
  • e5cc2a6360 docs(research): add qwen3.8-27b decode speed analysis haylan 2026-11-11 16:51:45 +01:00
  • 5b866b93ff feat(scripts): add redownload-model.sh to force-refresh a GGUF haylanandClaude Bot 2026-11-11 16:36:08 +01:00
  • 2585f3eaab feat(llm): re-enable RAM prompt cache (8192 MiB) with --no-cache-idle-slots haylanandClaude Bot 2026-11-06 19:24:28 +01:00
  • 51684fb7a4 feat(llm): add --spec-draft-p-min 0.5 to DFlash2 spec args haylanandClaude Bot 2026-11-06 18:53:27 +01:00
  • 6ce527741b chore(env): lower DFlash2 --spec-draft-n-max to 3 for long-context acceptance haylanandClaude Bot 2026-11-06 18:49:05 +01:00
  • 9fbdf3e664 Merge pull request 'Feat/dual gpu' (#67) from feat/dual-gpu into main haylan 2026-11-04 06:23:16 +00:00
  • 1484907ff8 docs(research): add inference engine, spec decoding and multi-GPU research haylanandClaude Bot 2026-11-02 19:42:45 +01:00
  • a89cccb3d5 chore(env): set LLAMA_CACHE_RAM to 0 in .env.example haylan 2026-10-31 19:04:40 +01:00
  • c25e7e835f chore(env): reduced LLAMA_CACHE_RAM value in .env.example haylan 2026-10-31 18:49:30 +01:00
  • 8927e440aa feat(llm): make spec decoding and KV cache type .env toggles haylanandClaude Bot 2026-10-31 18:22:44 +01:00
  • 7d142ed59d fix(models): correct DFlash2 Q8_0 drafter download URL haylanandClaude Bot 2026-10-31 17:01:27 +01:00
  • e4febb5beb fix(models): update LLAMA_DRAFT_MODEL_FILE and its URL to correct variant haylan 2026-10-31 16:59:34 +01:00
  • d5712d5fd4 chore(models): update LLAMA_DRAFT_MODEL_FILE and its URL to use BF16 variant haylan 2026-10-31 16:54:15 +01:00
  • 8144286946 refactor(models): merge downloaders into one, driven by models.yml haylanandClaude Bot 2026-10-31 16:48:58 +01:00
  • 28c102e0d9 feat(bench): add scripts/bench.sh wrapper for speed-bench runs haylanandClaude Bot 2026-10-31 16:45:27 +01:00
  • 68348aad72 feat(bench): add speed-bench tool service for drafter benchmarks haylanandClaude Bot 2026-10-31 16:43:06 +01:00
  • baf54002cd Update README.md haylan 2026-10-31 16:24:33 +01:00
  • 8cd33e2766 Create dual-r9700-speed-and-offload.md haylan 2026-10-31 16:24:31 +01:00
  • f4bc4dd7fa fix(llm): drop --cache-reuse, unsupported by this model's context haylanandClaude Bot 2026-10-20 21:12:07 +02:00
  • 408ed3e586 feat(llm): raise prompt cache RAM budget via --cache-ram haylanandClaude Bot 2026-10-20 21:05:27 +02:00
  • 433ebc6597 docs(research): mark opencode-cli-setup as v1-era haylanandClaude Bot 2026-10-05 21:36:27 +02:00
  • d9fc3418a1 docs(opencode): document v2 config, mark v1 deprecated haylanandClaude Bot 2026-10-05 21:32:58 +02:00
  • 17747694bd feat(llm): switch to DFlash2 speculative decoding, drop qwen-classifier haylanandClaude Bot 2026-10-05 21:28:55 +02:00
  • 1e3ad169c0 chore(llm): univyed token size for agent usage haylan 2026-10-03 14:06:56 +02:00
  • b74c0e0ccc chore(config): revisteded ai configuration for opencode haylan 2026-10-02 06:53:33 +00:00
  • 1989ec9383 reduced context of small modell haylan 2026-09-25 08:17:42 +02:00
  • a3437e9cc7 chore(ai): enalbeld mtp mode haylan 2026-09-25 08:13:23 +02:00
  • ea07daa3cc Merge pull request 'Docs cleanup: restructure, move code comments into docs, remove image generation' (#65) from docs/cleanup-and-restructure into main haylan 2026-09-24 20:01:30 +00:00
  • e27f5c03b6 docs(agents): add writing-docs guide and new-term procedure haylanandClaude Bot 2026-09-24 21:52:19 +02:00
  • 05effe5c88 chore: move code comments into docs, leave one-line pointers haylanandClaude Bot 2026-09-24 21:52:00 +02:00
  • a0e5e9704b docs: restructure around source-of-truth pages haylanandClaude Bot 2026-09-24 21:51:01 +02:00
  • 072eeb78c2 fix(qdrant): bind port to loopback only haylanandClaude Bot 2026-09-24 21:47:16 +02:00
  • fd55ce6669 chore: remove image generation, Open WebUI, Lazytainer and neo4j haylanandClaude Bot 2026-09-24 21:47:12 +02:00
  • 8ae32102f9 feat(open-webui): add chat+image frontend, bypass omniroute for comfyui haylanandClaude Bot 2026-09-22 23:28:44 +02:00
  • d09a99accd fix(comfyui): seed custom_nodes/input/output/user dirs under --base-directory haylanandClaude Bot 2026-09-22 23:09:27 +02:00
  • 9903b66094 fix(comfyui): correct HF subfolder paths for text encoder/VAE download haylanandClaude Bot 2026-09-22 23:04:07 +02:00
  • 78adbf4cdf fix(update.sh): remove stale downloader-fast call haylanandClaude Bot 2026-09-22 23:02:38 +02:00
  • 69d0d76d49 fix(comfyui): add missing launch command, drop dead PUID/PGID vars haylanandClaude Bot 2026-09-22 23:01:24 +02:00
  • 38ec636fb5 chore(scripts): mark switch-model.sh executable haylanandClaude Bot 2026-09-22 22:43:36 +02:00
  • db1ef90688 Merge remote-tracking branch 'origin/tool-cassefier-model' haylan 2026-09-22 22:39:21 +02:00
  • 4e310741c7 feat(comfyui): switch to Qwen-Image 2.1 uncensored GGUF model haylanandClaude Bot 2026-09-22 22:38:27 +02:00
  • a3f1099bfc docs(research): OmniRoute memory tools hijack classifier tool-call, x-omniroute-no-memory fix haylan 2026-09-15 23:31:58 +02:00
  • ed90256f43 docs(research): trace 48-min outage to third OmniRoute timeout and open ROCm#6630 GPU hang haylan 2026-09-15 23:31:58 +02:00
  • fa852c8fb4 docs(research): qwen-classifier needs no context-window match; 8B/30B won't fit VRAM haylan 2026-09-15 23:31:58 +02:00
  • edad6b2249 Merge branch 'research/opencode-auto-compact' haylan 2026-09-14 07:44:32 +02:00
  • 2250e804db Implement code changes to enhance functionality and improve performance haylan 2026-09-11 13:15:30 +02:00
  • e31647812a fix(omniroute): raise REQUEST_TIMEOUT_MS and enable --cache-reuse to stop non-ping SSE stream aborts haylanandClaude Bot 2026-09-10 10:10:40 +02:00
  • 930e407053 docs(llm): document qwen-classifier reality, add --reasoning off safety net haylanandClaude Bot 2026-09-09 19:10:32 +02:00
  • 76043e2c6f tune(llm): partial GPU offload for qwen-classifier, real headroom haylanandClaude Bot 2026-09-09 18:10:39 +02:00
  • 1102273384 fix(llm): enable flash-attn on qwen-classifier, real VRAM cause found haylanandClaude Bot 2026-09-09 17:04:05 +02:00
  • ba9ace6f71 fix(llm): shrink qwen-classifier's compute buffer for real VRAM headroom haylanandClaude Bot 2026-09-09 17:02:44 +02:00
  • 8e2650807c fix(llm): move qwen-classifier to GPU, right-size context haylanandClaude Bot 2026-09-09 16:59:02 +02:00
  • 4353e5c0e8 merge: follow-up fix for qwen-classifier model file haylanandClaude Bot 2026-09-09 16:38:11 +02:00
  • 20cc0bcc70 fix(llm): point qwen-classifier at the Q8 GGUF already on disk haylanandClaude Bot 2026-09-09 16:37:54 +02:00
  • c1e30ec9bb Merge pull request 'Fix/omniroute pr agent timeout' (#54) from fix/omniroute-pr-agent-timeout into main haylan 2026-09-09 14:29:13 +00:00
  • b3a64fe4b5 chroe(chore): added markdown for harnesses haylan 2026-09-09 16:28:42 +02:00
  • 828bd4c046 feat(llm): dedicate a CPU-only backend for the qwen-code tool-call classifier haylanandClaude Bot 2026-09-09 16:26:51 +02:00
  • 16df051318 docs(research): add ponytail audit report highlighting over-engineering and complexity issues haylan 2026-09-09 08:22:31 +02:00
  • 128503b68a docs: add AGENTS.md and QWEN.md with agent instructions and project overview haylan 2026-09-09 07:32:09 +02:00
  • df900404c0 chore(config): fix ignored path .qwen/temp to .qwen/tmp haylan 2026-09-08 12:46:32 +02:00
  • f4729ba704 docs(research): evaluate Colibrì for this stack haylan 2026-09-08 12:46:16 +02:00
  • 8f3feb4881 Merge branch 'feat-rag-databases' haylanandClaude Bot 2026-09-08 12:30:27 +02:00
  • 6f4e736da8 docs(coding-cli-setup): split per-CLI docs into their own files haylanandClaude Bot 2026-09-08 12:30:08 +02:00
  • 4c8a9c039e Merge pull request 'fix(update.sh): stop config sync loop from dying silently on a missing key' (#53) from feat-rag-databases into main haylan 2026-09-07 18:10:23 +00:00
  • 2ee308c1d9 Merge branch 'main' into feat-rag-databases haylan 2026-09-07 18:10:16 +00:00
  • 1fcf30e9a1 fix(update.sh): stop config sync loop from dying silently on a missing key haylanandClaude Bot 2026-09-07 20:09:31 +02:00
  • 31e9aab1f3 Merge pull request 'Add Qdrant/Neo4j RAG storage + update.sh gum fixes' (#52) from feat-rag-databases into main haylan 2026-09-07 18:07:48 +00:00
  • 665c3cb630 Merge branch 'main' into feat-rag-databases haylan 2026-09-07 18:07:42 +00:00
  • e151aa6ffe feat(update.sh): vendor gum binary for the R9700's offline install haylanandClaude Bot 2026-09-07 20:04:56 +02:00
  • feb7469f0b fix(update.sh): report why gum auto-install failed instead of failing silently haylanandClaude Bot 2026-09-07 20:01:20 +02:00
  • b51f7f9ad5 Merge pull request 'feat: add qdrant and neo4j for RAG vector/graph storage' (#50) from feat-rag-databases into main haylan 2026-09-07 17:56:49 +00:00
  • fbb949d417 Merge branch 'main' into feat-rag-databases haylan 2026-09-07 17:56:40 +00:00
  • 9767261a96 Merge pull request 'feat: remove llama-server-fast (Qwen3-4B classifier model)' (#51) from remove-fast-model into main haylan 2026-09-07 17:56:27 +00:00
  • 7a654ead91 chore: ignore .qwen/temp haylan 2026-09-07 19:54:38 +02:00
  • 2bfe6dbd29 docs: point knowledge.proxy-ai.home at Neo4j's browser haylanandClaude Bot 2026-09-07 19:40:51 +02:00
  • 5d6a17fd9b feat: remove llama-server-fast (Qwen3-4B classifier model) haylanandClaude Bot 2026-09-07 19:33:50 +02:00
  • 20f2ec3ab2 feat: add qdrant and neo4j for RAG vector/graph storage haylanandClaude Bot 2026-09-07 19:20:14 +02:00
  • 5b7548dc7c Merge pull request 'Fix llama-server-fast context-size exhaustion breaking Auto Mode' (#49) from fix-fastmodel-context-size into main haylan 2026-09-06 20:21:53 +00:00
  • ea7b05fb99 feat: interactive per-key conflict resolution in update.sh's config sync haylan 2026-09-06 22:20:27 +02:00
  • 9def240a8e feat: update.sh force-syncs tracked config from .env.example haylan 2026-09-06 22:10:51 +02:00
  • 52a92f6508 fix: llama-server-fast context-size exhaustion breaking Auto Mode classifier haylanandClaude Bot 2026-09-06 22:06:59 +02:00
  • 63938e95c9 Merge pull request 'Run git pull first in update.sh, not mid-script' (#48) from fix-update-sh-pull-order into main haylan 2026-09-06 19:40:10 +00:00
  • 7b6d3f5802 fix: run git pull first in update.sh, not mid-script haylan 2026-09-06 21:38:18 +02:00
  • 386a41200f Merge pull request 'Fix GPU pinned at 100% with two containers, flaky render group' (#47) from fix-gpu-pin-and-render-group into main haylan 2026-09-06 19:35:31 +00:00
  • 75033dacd7 fix: GPU pinned at 100% with two llama.cpp containers, flaky render group haylanandClaude Bot 2026-09-06 21:33:40 +02:00
  • 1ee2e76033 Merge pull request 'Downloader for Qwen-Image weights, switch-model.sh' (#46) from comfyui-model-and-switch-script into main haylan 2026-09-06 19:01:02 +00:00
  • 4b47a1769d feat: downloader for Qwen-Image weights, switch-model.sh script haylanandClaude Bot 2026-09-06 20:57:38 +02:00
  • 1932981f09 Merge pull request 'Add llama-server-fast: small non-thinking classifier/fast model' (#45) from add-fast-model into main haylan 2026-09-06 18:39:30 +00:00
  • d984c10835 feat: add llama-server-fast, a small non-thinking classifier model haylanandClaude Bot 2026-09-06 20:24:59 +02:00
  • 71c9003bd8 Create dashscope-websearch-selfhost-options.md haylan 2026-09-06 16:10:26 +02:00
  • ed83fca05c feat(comfyui): add local image-gen service (#41) haylanandClaude Bot 2026-09-05 22:40:07 +02:00
  • 7d1ff2f54f docs(research): add image-gen backend survey and omniroute/qwen websearch notes haylanandClaude Bot 2026-09-05 21:24:23 +02:00
  • ac3f730f83 docs(research): recommend Qwen-Image FP8 for full-VRAM diffusion build haylanandClaude Bot 2026-09-05 21:03:13 +02:00
  • 451d5c7b28 docs(research): confirm lazytainer/omniroute idle-stop conflict (#40) haylanandClaude Bot 2026-09-05 20:57:24 +02:00
  • d8736b6dd7 fix(llama-server): quantize KV cache, restore full 262144 context haylanandClaude Bot 2026-09-05 12:00:07 +02:00
  • 5767f548c3 perf(llama-server): enable flash attention haylanandClaude Bot 2026-09-05 11:32:17 +02:00
  • 633292b291 fix(omniroute): raise stream-idle timeout to 180s haylanandClaude Bot 2026-09-05 11:29:12 +02:00
  • 23e90fe8fb fix(llama-server): cap concurrent slots at 2 to curb prefill contention haylanandClaude Bot 2026-09-05 11:26:30 +02:00
  • ae812cd9e0 feat(ctx): larger context size haylan 2026-09-04 21:47:38 +02:00