docs: research Qwen3.8-27B Unsloth dynamic GGUF quant + VRAM budget

Resolves the wayfinder research question: unsloth/Qwen3.8-27B-GGUF ships
UD-Q4_K_XL.gguf (17.6GB) directly. Documents VRAM footprint on 32GB
RDNA4 (R9700/gfx1201) at 32K and 128K context via llama.cpp ROCm/HIP.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

docs/research/qwen3.8-27b-quant.md

@@ -0,0 +1,128 @@
+# Research: Unsloth dynamic 4-bit GGUF for Qwen/Qwen3.8-27B
+
+**Question:** Does a UD-Q4_K_XL (or equivalent Unsloth dynamic) GGUF quant of
+`Qwen/Qwen3.8-27B` exist? If not, what's the closest tool-call-safe
+alternative, and what's the VRAM footprint on a 32GB RDNA4 card
+(AMD Radeon AI PRO R9700, gfx1201) via llama.cpp ROCm/HIP, at 32K and 128K
+context?
+
+**Answer: Yes, it exists.** `unsloth/Qwen3.8-27B-GGUF` on Hugging Face
+contains `Qwen3.8-27B-UD-Q4_K_XL.gguf` (17.6 GB), Unsloth's Dynamic v3.0
+4-bit "XL" quant that keeps higher precision on attention/critical layers.
+This is the recommended download — no need to fall back to AWQ or plain
+Q4_K_M.
+
+## Model identity (context)
+
+`Qwen/Qwen3.8-27B` is a real, current model — released 2026-08-14 by
+Alibaba's Qwen team (Apache 2.0), the successor to Qwen3.6-27B. It's a dense
+27B hybrid model: 64 layers, `full_attention_interval: 4`, meaning only 16
+of 64 layers are standard full-attention (KV-caching) layers; the other 48
+are Gated DeltaNet linear-attention layers whose state is fixed-size
+regardless of context length. `hidden_size=5120`, `num_attention_heads=24`,
+`num_key_value_heads=4` (GQA), `head_dim=256`, native context 262,144 tokens
+(YaRN-extensible to 1M). It's natively multimodal (vision + video) via a
+separate `mmproj` file.
+Source: [Qwen/Qwen3.8-27B model card](https://huggingface.co/Qwen/Qwen3.8-27B),
+[Qwen/Qwen3.8-27B config.json](https://huggingface.co/Qwen/Qwen3.8-27B/raw/main/config.json),
+[QwenLM/Qwen3.8 GitHub repo](https://github.com/QwenLM/Qwen3.8).
+
+## Recommended repo / file
+
+- **Repo:** `unsloth/Qwen3.8-27B-GGUF`
+- **File:** `Qwen3.8-27B-UD-Q4_K_XL.gguf` — **17.6 GB**
+- Optional vision projector (only needed for image/video input):
+  `mmproj-BF16.gguf` (931 MB) or `mmproj-F16.gguf` (928 MB)
+
+Confirmed directly from the repo's file tree (primary source, fetched from
+the HF repo page itself):
+[unsloth/Qwen3.8-27B-GGUF/tree/main](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/tree/main).
+The repo also ships `UD-Q4_K_M` (16.5 GB), `UD-Q4_K_S` (15.4 GB), and other
+Dynamic v3.0 sizes (IQ1 through Q8_K_XL), but `UD-Q4_K_XL` is the one that
+matches the "dynamic 4-bit XL" spec in the ticket and gives the best
+accuracy/size tradeoff at 4-bit — Unsloth's own docs name it as the
+recommended default for this model.
+Source: [Unsloth docs — Qwen3.8 how-to-run page](https://unsloth.ai/docs/models/qwen3.8)
+("Unsloth recommends UD-Q4_K_XL as the primary quantization... 4-bit quants
+work on 16-19GB VRAM").
+
+Why prefer this over plain `Q4_K_M`/AWQ for tool-calling fidelity: Unsloth's
+Dynamic v3.0 quantization selectively keeps higher bit-width on
+attention/critical layers rather than uniformly quantizing everything, and
+Unsloth's own model-card notes call out improved "tool calling: improved
+parsing [of] nested objects to make tools succeed more" for this quant
+family. Since a dynamic XL quant is directly available, there was no need to
+fall back to evaluating AWQ alternatives.
+
+**Confidence note:** The repo tree listing and file sizes were fetched
+directly from the Hugging Face page (primary source) and are high
+confidence. I did not download and independently checksum the file. The
+"improved tool calling" claim comes from Unsloth's own docs page (primary,
+first-party), not an independent benchmark — treat it as a vendor claim, not
+independently verified.
+
+## ModelScope / other sources checked
+
+Search results also surfaced `Qwen/Qwen3.8-27B` mirrored patterns typical of
+ModelScope + HF dual-publishing (per the QwenLM GitHub repo, which states
+new releases go out on "Hugging Face and ModelScope" simultaneously), and a
+non-Unsloth GGUF from `bartowski/Qwen3.8-27B-GGUF` (imatrix-calibrated,
+includes tool-calling/reasoning conversations in the calibration corpus) as
+a secondary alternative if the Unsloth repo were ever unavailable. I did not
+deep-dive ModelScope's page directly (Unsloth's HF repo already answers the
+question), so treat the ModelScope-mirror claim as **unconfirmed** —
+inferred from the GitHub repo's release notes rather than fetched directly
+from modelscope.cn.
+
+## VRAM footprint on a 32GB RDNA4 card (R9700, gfx1201) via llama.cpp ROCm/HIP
+
+**Weights:** 17.6 GB (`UD-Q4_K_XL.gguf`, fits entirely on-GPU on a 32GB card).
+
+**KV cache:** Because only 16 of 64 layers are standard full-attention
+(GQA: 4 KV heads × 256 head_dim), the KV cache scales with just those 16
+layers — the other 48 Gated-DeltaNet layers carry a fixed, context-length-independent
+recurrent state (order tens of MB total, negligible next to the attention KV cache).
+
+Per-token KV cache (fp16, both K and V, 16 full-attn layers):
+`16 layers × 2 (K+V) × 4 kv_heads × 256 head_dim × 2 bytes = 64 KiB/token`
+
+| Context | KV cache (fp16) | Weights | Weights + KV cache | Headroom on 32GB |
+|---|---|---|---|---|
+| 32K tokens | ~2.0 GB | 17.6 GB | **~19.6 GB** | ~12 GB free (compute buffer, batch, overhead) |
+| 128K tokens | ~8.0 GB | 17.6 GB | **~25.6 GB** | ~6 GB free — comfortable but tighter |
+
+Both fit on a 32GB card with room to spare. If more headroom is wanted at
+128K (e.g., for larger batch/ubatch or multiple parallel sequences),
+quantizing the KV cache to Q8_0 in llama.cpp (`--cache-type-k q8_0
+--cache-type-v q8_0`) roughly halves the cache to ~4 GB, bringing the 128K
+total to ~21.6 GB.
+
+**Sanity check against a third-party source:** a blog doing the same math
+independently states the full 262,144-token native context costs "about
+17GB of KV cache" for this hybrid architecture. My formula gives
+262,144 × 64 KiB ≈ 16.0 GB, consistent with that figure.
+Source: [WebSearch summary citing "Qwen3.8-27B VRAM/KV-cache math" blog](https://umesh-malik.com/blog/qwen3-8-27b-vram-kv-cache-math)
+(secondary source, used only as a cross-check on the arithmetic, not as a
+primary claim).
+
+**ROCm/HIP + gfx1201 support:** ROCm 7.2 added official `gfx1201` support
+(some report 7.1.1 working via `HSA_OVERRIDE_GFX_VERSION`); llama.cpp builds
+with `-DGGML_HIP=ON -DCMAKE_HIP_ARCHITECTURES=gfx1201`. Note: community
+reports say the Vulkan backend is currently ~23% faster than ROCm/HIP on
+gfx1201 for token generation, and the HIP backend has a known issue keeping
+the GPU at elevated clocks/power after idle. These are performance/operational
+notes, not blockers for fitting the model in VRAM.
+Sources: [tlee933/llama.cpp-rdna4-gfx1201 GitHub](https://github.com/tlee933/llama.cpp-rdna4-gfx1201),
+[ggml-org/llama.cpp discussion #20881 (R9700 ROCm)](https://github.com/ggml-org/llama.cpp/discussions/20881),
+[ggml-org/llama.cpp discussion #15021 (ROCm perf)](https://github.com/ggml-org/llama.cpp/discussions/15021).
+**Confidence note:** these ROCm/gfx1201 specifics come from secondary
+(community) sources, not AMD's or ggml-org's official docs directly fetched
+— treat performance numbers as indicative, not guaranteed.
+
+## Bottom line for the wayfinder map
+
+- Use **`unsloth/Qwen3.8-27B-GGUF`**, file **`Qwen3.8-27B-UD-Q4_K_XL.gguf`** (17.6 GB).
+- Fits comfortably on the 32GB R9700 at both 32K context (~19.6 GB total)
+  and 128K context (~25.6 GB total, or ~21.6 GB with q8_0 KV cache).
+- Build llama.cpp with ROCm/HIP targeting `gfx1201`; Vulkan is a viable
+  faster alternative backend if HIP's idle-power quirk is a problem.