diff --git a/docs/server-planing.md b/docs/server-planing.md new file mode 100644 index 0000000..f3f6b20 --- /dev/null +++ b/docs/server-planing.md @@ -0,0 +1,108 @@ +# 4x Radeon AI PRO R9700 — Rackmount LLM Server Build Plan + +**Goal:** Local LLM inference/training rig running 3-4x AMD Radeon AI PRO R9700 (32GB) GPUs, rack-mounted in a full-size 19" rack. + +--- + +## GPU + +| Spec | Value | +|---|---| +| Model | AMD Radeon AI PRO R9700 32GB | +| VRAM | 32GB GDDR6, 256-bit bus | +| Interface | PCIe 5.0 x16 (backward-compatible with PCIe 4.0 slots at x16 bandwidth) | +| Form factor | Dual-slot, blower-style cooler | +| TDP | 300W | +| Quantity | 4x → ~1,200W GPU load alone | + +Blower coolers exhaust heat out the back of the case rather than into it — important for stacking 4 cards close together. + +--- + +## Motherboard + +**Gigabyte MZ32-AR0 (Rev 1.0)** + +| Spec | Value | +|---|---| +| Socket | Single SP3, EPYC 7002/7001 series | +| Form factor | E-ATX (305mm x 330mm) | +| PCIe slots | 4x full PCIe Gen4 x16 (Slot_3, Slot_4, Slot_6, Slot_7) + additional Gen4 x8 / Gen3 slots | +| Memory | 8-channel DDR4 RDIMM/LRDIMM, 16 DIMM slots | +| Management | ASPEED AST2500 BMC (IPMI) | + +**Note:** CPU/memory socket sits close to the PCIe slots — plan for PCIe riser cables to give the R9700's clearance and to route to rear chassis slot cutouts. + +--- + +## CPU + +**AMD EPYC 7282** (Rome / 2nd Gen, SP3) + +- 16 cores — sufficient since the workload is GPU-bound, not CPU-bound +- Full access to Rome's 128 PCIe lanes on this single-socket board +- Budget-friendly used-market option vs. Threadripper PRO + +--- + +## Memory + +- **Type:** DDR4 RDIMM or LRDIMM only — unbuffered (UDIMM) will not POST on SP3 +- Size to your model/dataset needs; 8-channel config gives strong bandwidth for CPU-side preprocessing and system overhead + +--- + +## Power Supply + +- Estimated system draw: ~1,200W (GPUs) + CPU + overhead → **1,600-2,000W** PSU needed +- At 4U rack size, use a **redundant CRPS-style server PSU** rather than a single consumer ATX unit — better suited to 24/7 operation + +--- + +## Chassis + +**4U rackmount, E-ATX compatible, full-depth (420mm+)** + +Candidates: +- Rosewill RSV-L4620-class (supports up to 4 GPUs, E-ATX compatible, hot-swap bays) +- Rackchoice/RackOwl 4U E-ATX (7 full-height expansion slots, 420mm depth) + +Requirements: +- Front-to-back airflow (matches R9700 blower exhaust direction) +- 7 full-height PCIe slot cutouts at rear +- Riser card support if GPU clearance near CPU socket is tight + +--- + +## Rack + +**42U full-size rack, 19" width, ≥1,000-1,100mm usable depth** + +| Item | Space | +|---|---| +| GPU server (4U chassis) | 4U | +| UPS | ~2-3U | +| Network switch | 1U | +| Cable management / PDU | 1-2U | +| **Used now** | **~8-10U** | +| **Free for expansion** | **~32-34U** | + +- Choose an **enclosed rack** (side panels + front/rear doors) for dust control and directed airflow, appropriate for a dedicated server room/garage. +- Confirm listed depth explicitly supports full GPU servers — many "42U" racks are sized for shallow network gear, not 420mm+ deep GPU chassis. + +--- + +## Sourcing links + +- [Netzwerkschrank, 37HE 600x1000](https://www.ebay.de/itm/314598915865?var=613137180576) — candidate rack; maybe go taller than 37HE +- [Gigabyte MZ32-AR0 Rev1.0 + EPYC 7282 16-Kern 2.8GHz](https://www.ebay.de/itm/358886890399) — mainboard+CPU bundle matching the spec above. Still needs RAM — prefer older/used DDR4 RDIMM over new, it's cheaper. + +--- + +## Open Items / Next Steps + +- [ ] Verify current ROCm/Linux driver compatibility for R9700 on Rome-era (SP3) PCIe topology +- [ ] Confirm riser cable part numbers/compatibility with MZ32-AR0 + chosen 4U chassis +- [ ] Size UPS capacity against full system load (GPU server + switch + growth headroom) +- [ ] Source EPYC 7282 + MZ32-AR0 (used/refurb market) +- [ ] Confirm chassis rail kit compatibility with target rack