diff --git a/docs/operations.md b/docs/operations.md index 7e216ee..ffc396a 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -1,6 +1,6 @@ # Operations -> **Source of truth:** [`scripts/update.sh`](../scripts/update.sh), [`.env.example`](../.env.example), [`models.yml`](../models.yml), and the `speed-bench` service in [`docker-compose.yml`](../docker-compose.yml) with its [`speed-bench/Dockerfile`](../speed-bench/Dockerfile) and [`scripts/bench.sh`](../scripts/bench.sh). This page explains *what the script does and why*; the steps and defaults live in the source. If they disagree, the source wins. +> **Source of truth:** [`scripts/update.sh`](../scripts/update.sh), [`scripts/redownload-model.sh`](../scripts/redownload-model.sh), [`.env.example`](../.env.example), [`models.yml`](../models.yml), and the `speed-bench` service in [`docker-compose.yml`](../docker-compose.yml) with its [`speed-bench/Dockerfile`](../speed-bench/Dockerfile) and [`scripts/bench.sh`](../scripts/bench.sh). This page explains *what the script does and why*; the steps and defaults live in the source. If they disagree, the source wins. The stack runs on the **server**. After any change to the repo, run `./scripts/update.sh` there: it is the one command that brings the running stack in sync. It's safe to re-run at any time, and it only recreates what changed. @@ -45,6 +45,13 @@ One `downloader` service in the `tools` compose profile fetches every file liste - **Files live in the named `models` volume**, not a host bind-mount. The downloader runs as root because the named volume is created root-owned. - **It needs internet on each run**: it installs `curl` and `yq` from Alpine's package mirror before reading the list. +### Forcing a model re-download + +The skip check means `update.sh` never picks up an upstream change to a file that's already on disk (for example, a re-uploaded GGUF with the same name). [`scripts/redownload-model.sh`](../scripts/redownload-model.sh) forces it: with no argument it refreshes `LLAMA_MODEL_FILE` from `.env`, or pass a `file` name from `models.yml` (for example the drafter). It reuses the compose downloader, so the URL still lives only in `models.yml`. + +- The old file is renamed to `.old` and deleted only after the new download succeeds. If the download fails or is interrupted, the old file is put back. This needs room for both copies in the `models` volume at once. +- It ends by restarting `llama-server`, because the running process keeps the old file mapped until it restarts. + ## Speed bench `speed-bench` runs llama.cpp's [SPEED-Bench client](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/bench/speed-bench/README.md) against the running `llama-server`. It measures decode speed (`avg_pred_t/s`), latency and draft acceptance per category. Use it to compare drafters and their flags with numbers instead of gut feel. It's in the `tools` profile, so `update.sh` never builds or runs it. The first run builds the image. diff --git a/scripts/redownload-model.sh b/scripts/redownload-model.sh new file mode 100755 index 0000000..fd4189d --- /dev/null +++ b/scripts/redownload-model.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +# Forces a fresh download of a model file listed in models.yml, replacing the one on disk. +# Why and how: docs/operations.md#forcing-a-model-re-download +set -euo pipefail +cd "$(dirname "$0")/.." + +if [ $# -gt 1 ] || [ "${1:-}" = "-h" ] || [ "${1:-}" = "--help" ]; then + echo "Usage: $0 [model-file] (default: LLAMA_MODEL_FILE from .env)" >&2 + exit 1 +fi + +file="${1:-}" +if [ -z "$file" ]; then + [ -f .env ] || { echo ".env missing, run scripts/update.sh first" >&2; exit 1; } + file="$(grep -m1 '^LLAMA_MODEL_FILE=' .env | cut -d= -f2- | tr -d '\r"'"'" || true)" + [ -n "$file" ] || { echo "LLAMA_MODEL_FILE is not set in .env" >&2; exit 1; } +fi +grep -qxF " - file: $file" models.yml || { echo "$file is not listed in models.yml" >&2; exit 1; } + +# Runs a snippet against the models volume; $1 in it is the model file. +in_volume() { docker compose --profile tools run --rm --entrypoint sh downloader -c "cd /models; $1" _ "$file"; } + +# Put the previous file back if the download didn't finish, so a failed run never loses the working model. +ok=false +restore() { + $ok || in_volume '[ ! -f "$1.old" ] || { rm -f "$1" "$1.part"; mv "$1.old" "$1"; }' || true +} +trap restore EXIT + +echo "==> moving aside the current $file (kept until the new download succeeds)" +in_volume '[ -f "$1" ] || { [ ! -f "$1.old" ] || mv "$1.old" "$1"; }; [ ! -f "$1" ] || mv -f "$1" "$1.old"; rm -f "$1.part"' + +echo "==> downloading $file" +docker compose --profile tools run --rm downloader + +ok=true +in_volume 'rm -f "$1.old"' + +echo "==> restarting llama-server to load the new file" +docker compose restart llama-server