haylan/LLM-Server
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(bench): add speed-bench tool service for drafter benchmarks

Browse Source 
Runs llama.cpp's SPEED-Bench client (pinned commit) against
llama-server over the compose network, in the tools profile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
haylanandClaude Bot committed 2026-10-31 16:43:06 +01:00
1 parent f4bc4dd7fa
commit 68348aad72
5 files changed
+45 -3
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

speed-bench/Dockerfile
+9
View File
@@ -0,0 +1,9 @@
# see docs/operations.md#speed-bench
FROM python:3.12-slim
ARG LLAMA_CPP_REF=0ec4027a2a0183de87c007069adddc06e665f7de
ADD https://raw.githubusercontent.com/ggml-org/llama.cpp/${LLAMA_CPP_REF}/tools/server/bench/speed-bench/requirements.txt \
https://raw.githubusercontent.com/ggml-org/llama.cpp/${LLAMA_CPP_REF}/tools/server/bench/speed-bench/speed_bench.py \
https://raw.githubusercontent.com/ggml-org/llama.cpp/${LLAMA_CPP_REF}/tools/server/bench/speed-bench/speed_bench_compare.py \
/app/
RUN pip install --no-cache-dir -r /app/requirements.txt
WORKDIR /results