# Research: Unsloth dynamic 4-bit GGUF for Qwen/Qwen3.8-27B

**Question:** Does a UD-Q4_K_XL (or equivalent Unsloth dynamic) GGUF quant of
`Qwen/Qwen3.8-27B` exist? If not, what's the closest tool-call-safe
alternative, and what's the VRAM footprint on a 32GB RDNA4 card
(AMD Radeon AI PRO R9700, gfx1201) via llama.cpp ROCm/HIP, at 32K and 128K
context?

**Answer: Yes, it exists.** `unsloth/Qwen3.8-27B-GGUF` on Hugging Face
contains `Qwen3.8-27B-UD-Q4_K_XL.gguf` (17.6 GB), Unsloth's Dynamic v3.0
4-bit "XL" quant that keeps higher precision on attention/critical layers.
This is the recommended download — no need to fall back to AWQ or plain
Q4_K_M.

## Model identity (context)

`Qwen/Qwen3.8-27B` is a real, current model — released 2026-08-14 by
Alibaba's Qwen team (Apache 2.0), the successor to Qwen3.6-27B. It's a dense
27B hybrid model: 64 layers, `full_attention_interval: 4`, meaning only 16
of 64 layers are standard full-attention (KV-caching) layers; the other 48
are Gated DeltaNet linear-attention layers whose state is fixed-size
regardless of context length. `hidden_size=5120`, `num_attention_heads=24`,
`num_key_value_heads=4` (GQA), `head_dim=256`, native context 262,144 tokens
(YaRN-extensible to 1M). It's natively multimodal (vision + video) via a
separate `mmproj` file.
Source: [Qwen/Qwen3.8-27B model card](https://huggingface.co/Qwen/Qwen3.8-27B),
[Qwen/Qwen3.8-27B config.json](https://huggingface.co/Qwen/Qwen3.8-27B/raw/main/config.json),
[QwenLM/Qwen3.8 GitHub repo](https://github.com/QwenLM/Qwen3.8).

## Recommended repo / file

- **Repo:** `unsloth/Qwen3.8-27B-GGUF`
- **File:** `Qwen3.8-27B-UD-Q4_K_XL.gguf` — **17.6 GB**
- Optional vision projector (only needed for image/video input):
  `mmproj-BF16.gguf` (931 MB) or `mmproj-F16.gguf` (928 MB)

Confirmed directly from the repo's file tree (primary source, fetched from
the HF repo page itself):
[unsloth/Qwen3.8-27B-GGUF/tree/main](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/tree/main).
The repo also ships `UD-Q4_K_M` (16.5 GB), `UD-Q4_K_S` (15.4 GB), and other
Dynamic v3.0 sizes (IQ1 through Q8_K_XL), but `UD-Q4_K_XL` is the one that
matches the "dynamic 4-bit XL" spec in the ticket and gives the best
accuracy/size tradeoff at 4-bit — Unsloth's own docs name it as the
recommended default for this model.
Source: [Unsloth docs — Qwen3.8 how-to-run page](https://unsloth.ai/docs/models/qwen3.8)
("Unsloth recommends UD-Q4_K_XL as the primary quantization... 4-bit quants
work on 16-19GB VRAM").

Why prefer this over plain `Q4_K_M`/AWQ for tool-calling fidelity: Unsloth's
Dynamic v3.0 quantization selectively keeps higher bit-width on
attention/critical layers rather than uniformly quantizing everything, and
Unsloth's own model-card notes call out improved "tool calling: improved
parsing [of] nested objects to make tools succeed more" for this quant
family. Since a dynamic XL quant is directly available, there was no need to
fall back to evaluating AWQ alternatives.

**Confidence note:** The repo tree listing and file sizes were fetched
directly from the Hugging Face page (primary source) and are high
confidence. I did not download and independently checksum the file. The
"improved tool calling" claim comes from Unsloth's own docs page (primary,
first-party), not an independent benchmark — treat it as a vendor claim, not
independently verified.

## ModelScope / other sources checked

Search results also surfaced `Qwen/Qwen3.8-27B` mirrored patterns typical of
ModelScope + HF dual-publishing (per the QwenLM GitHub repo, which states
new releases go out on "Hugging Face and ModelScope" simultaneously), and a
non-Unsloth GGUF from `bartowski/Qwen3.8-27B-GGUF` (imatrix-calibrated,
includes tool-calling/reasoning conversations in the calibration corpus) as
a secondary alternative if the Unsloth repo were ever unavailable. I did not
deep-dive ModelScope's page directly (Unsloth's HF repo already answers the
question), so treat the ModelScope-mirror claim as **unconfirmed** —
inferred from the GitHub repo's release notes rather than fetched directly
from modelscope.cn.

## VRAM footprint on a 32GB RDNA4 card (R9700, gfx1201) via llama.cpp ROCm/HIP

**Weights:** 17.6 GB (`UD-Q4_K_XL.gguf`, fits entirely on-GPU on a 32GB card).

**KV cache:** Because only 16 of 64 layers are standard full-attention
(GQA: 4 KV heads × 256 head_dim), the KV cache scales with just those 16
layers — the other 48 Gated-DeltaNet layers carry a fixed, context-length-independent
recurrent state (order tens of MB total, negligible next to the attention KV cache).

Per-token KV cache (fp16, both K and V, 16 full-attn layers):
`16 layers × 2 (K+V) × 4 kv_heads × 256 head_dim × 2 bytes = 64 KiB/token`

| Context | KV cache (fp16) | Weights | Weights + KV cache | Headroom on 32GB |
|---|---|---|---|---|
| 32K tokens | ~2.0 GB | 17.6 GB | **~19.6 GB** | ~12 GB free (compute buffer, batch, overhead) |
| 128K tokens | ~8.0 GB | 17.6 GB | **~25.6 GB** | ~6 GB free — comfortable but tighter |

Both fit on a 32GB card with room to spare. If more headroom is wanted at
128K (e.g., for larger batch/ubatch or multiple parallel sequences),
quantizing the KV cache to Q8_0 in llama.cpp (`--cache-type-k q8_0
--cache-type-v q8_0`) roughly halves the cache to ~4 GB, bringing the 128K
total to ~21.6 GB.

**Sanity check against a third-party source:** a blog doing the same math
independently states the full 262,144-token native context costs "about
17GB of KV cache" for this hybrid architecture. My formula gives
262,144 × 64 KiB ≈ 16.0 GB, consistent with that figure.
Source: [WebSearch summary citing "Qwen3.8-27B VRAM/KV-cache math" blog](https://umesh-malik.com/blog/qwen3-8-27b-vram-kv-cache-math)
(secondary source, used only as a cross-check on the arithmetic, not as a
primary claim).

**ROCm/HIP + gfx1201 support:** ROCm 7.2 added official `gfx1201` support
(some report 7.1.1 working via `HSA_OVERRIDE_GFX_VERSION`); llama.cpp builds
with `-DGGML_HIP=ON -DCMAKE_HIP_ARCHITECTURES=gfx1201`. Note: community
reports say the Vulkan backend is currently ~23% faster than ROCm/HIP on
gfx1201 for token generation, and the HIP backend has a known issue keeping
the GPU at elevated clocks/power after idle. These are performance/operational
notes, not blockers for fitting the model in VRAM.
Sources: [tlee933/llama.cpp-rdna4-gfx1201 GitHub](https://github.com/tlee933/llama.cpp-rdna4-gfx1201),
[ggml-org/llama.cpp discussion #20881 (R9700 ROCm)](https://github.com/ggml-org/llama.cpp/discussions/20881),
[ggml-org/llama.cpp discussion #15021 (ROCm perf)](https://github.com/ggml-org/llama.cpp/discussions/15021).
**Confidence note:** these ROCm/gfx1201 specifics come from secondary
(community) sources, not AMD's or ggml-org's official docs directly fetched
— treat performance numbers as indicative, not guaranteed.

## Bottom line for the wayfinder map

- Use **`unsloth/Qwen3.8-27B-GGUF`**, file **`Qwen3.8-27B-UD-Q4_K_XL.gguf`** (17.6 GB).
- Fits comfortably on the 32GB R9700 at both 32K context (~19.6 GB total)
  and 128K context (~25.6 GB total, or ~21.6 GB with q8_0 KV cache).
- Build llama.cpp with ROCm/HIP targeting `gfx1201`; Vulkan is a viable
  faster alternative backend if HIP's idle-power quirk is a problem.