Building llamacpp on Windows w/ ROCm¶

Nightmare scenario: trying to build llamacpp from source with rocm on WINDOWS 😱

Why do this?¶

If you want llamacpp support with ROCm, you can just download the lemonade-sdk/llamacpp-rocm release which already includes support, however I want to apply a set of patches to llamacpp, so we need to build from scratch.

My Config for Reference¶

• Windows 11 (25H2) (Build 26200.9168)
• RX 7900XT (gfx1100 / RDNA3)
• AMD Adrenalin 26.8.1

Pre-requisites¶

Im specifically NOT following AMD's ROCm install guide because I want to use uv

• x64 Native Tools Command Prompt for VS 2022 (shell)
  • Download Visual Studio and install the "Desktop development with C++" Workload
  • Optional Packages Required:
    • MVSC v143 - VS 2022 C++ x64/x86 build tools (Latest) is checked
    • Windows 11 SDK (10.0.26100.7705)
    • C++ CMake tools for Windows
    • Testing tools core features - Build Tools
    • C++ AddressSanitizer
    • vcpkg package manager
• uv

References¶

• AMD ROCm Docs
• rocm wheels
• lemonade-sdk/llamacpp-rocm build docs
• hatedabamboo notes blogpost
• stew675/llama-cpp-rdna-boosts patches

Steps¶

1) Install ROCm via uv¶

Create a dedicated virtual environment and install the ROCm wheel from AMD's nightly index:

# From your project root (e.g. C:\Code\llamacpp-rocm)
uv venv .venv
.\.venv\Scripts\activate

uv pip install --index-url https://nightly.repo.amd.com/rocm/whl-next/ "rocm[libraries,devel,device-gfx1100]"

flags:

• libraries — runtime DLLs
• devel — headers, .lib import libs, cmake config files (this gives you _rocm_sdk_devel)
• device-gfx1100 — device-specific code for your GPU

example output as of Sept 2026:

Resolved 5 packages in 1.40s
Installed 5 packages in 525ms
 + rocm==10.1.0a20260907
 + rocm-sdk-core==10.1.0a20260907
 + rocm-sdk-devel==10.1.0a20260907
 + rocm-sdk-device-gfx1100==10.1.0a20260907
 + rocm-sdk-libraries==10.1.0a20260907

Then initialize the devel tree:

rocm-sdk init      # expands/links the devel tree into _rocm_sdk_devel
rocm-sdk targets   # confirm gfx1100 is in the supported list

2) Open the Correct Terminal¶

You must use the right shell:

• Open the x64 Native Tools Command Prompt for VS 2022 shell

Then switch from cmd to powershell, and activate your venv:

powershell
cd C:\Code\llamacpp-rocm
.\.venv\Scripts\Activate.ps1

3) Clone Both (options) Repo(s)¶

You can download llamacpp only here if you'd like

cd C:\Code\llamacpp-rocm

git clone https://github.com/ggml-org/llama.cpp
git clone https://github.com/stew675/llama-cpp-rdna-boosts

4) Apply the rdna-boosts Patches (optional)¶

cd llama.cpp

git apply ..\llama-cpp-rdna-boosts\rdna-boosts-all.patch

git status
# Should show modified files

5) Build with ROCm¶

5a) Set up paths and environment¶

# Resolve ROCm paths via the rocm-sdk CLI
$rocmRoot  = (rocm-sdk path --root)  -replace '\\','/'
$rocmCmake = (rocm-sdk path --cmake) -replace '\\','/'

$llvmBin = "$rocmRoot/lib/llvm/bin"

$env:PATH = "$llvmBin;$env:PATH"

# ROCm runtime tuning
# With the current version of ROCm I was getting issues with amdhip64_7.dll before adding these
# https://github.com/ROCm/TheRock/issues/7727
# https://github.com/JM00NJ/amdhip64-hipLaunchKernel-crash-fix
# https://github.com/ggml-org/llama.cpp/issues/17429
# Do some research on your card and adjust accordingly
$env:KMP_DUPLICATE_LIB_OK = "TRUE"
$env:HSA_OVERRIDE_GFX_VERSION = "11.0.0"
$env:HSA_ENABLE_SDMA = "0"

# Sanity check — should print clang.exe and clang++.exe
Get-ChildItem $llvmBin -Filter "clang*.exe" | Where-Object { $_.Name -in @("clang.exe","clang++.exe") }

5b) Configure¶

Verify DGPU_TARGETS and DCMAKE_RC_COMPILER!!

# use this default cmake build command if using vanilla llamacpp
cmake -S . -B build -G Ninja `
  -DGGML_HIP=ON `
  -DGPU_TARGETS=gfx1100 `
  -DGGML_HIP_ROCWMMA_FATTN=ON `
  -DCMAKE_C_COMPILER="$llvmBin/clang.exe" `
  -DCMAKE_CXX_COMPILER="$llvmBin/clang++.exe" `
  -DCMAKE_RC_COMPILER="C:/Program Files (x86)/Windows Kits/10/bin/10.0.26100.0/x64/rc.exe" `
  -DCMAKE_BUILD_TYPE=Release `
  -DLLAMA_CURL=OFF `
  -DCMAKE_PREFIX_PATH="$rocmCmake" `
  -DCMAKE_CXX_FLAGS="--rocm-device-lib-path=$rocmRoot/lib/llvm/amdgcn/bitcode"

# cmake command used when applying patches
cmake -S . -B build -G Ninja `
  -DGGML_HIP=ON `
  -DGPU_TARGETS=gfx1100 `
  -DGGML_HIP_ROCWMMA_FATTN=ON `
  -DGGML_HIP_GRAPHS=ON `
  -DGGML_NATIVE=ON `
  -DHIP_PLATFORM=amd `
  -DCMAKE_C_COMPILER="$llvmBin/clang.exe" `
  -DCMAKE_CXX_COMPILER="$llvmBin/clang++.exe" `
  -DCMAKE_RC_COMPILER="C:/Program Files (x86)/Windows Kits/10/bin/10.0.26100.0/x64/rc.exe" `
  -DCMAKE_BUILD_TYPE=Release `
  -DLLAMA_CURL=OFF `
  -DCMAKE_PREFIX_PATH="$rocmCmake" `
  -DCMAKE_CXX_FLAGS="--rocm-device-lib-path=$rocmRoot/lib/llvm/amdgcn/bitcode -mllvm --amdgpu-unroll-threshold-local=600"

5c) Build¶

cmake --build build --config Release -j

6) Copy ROCm Runtime Libraries to build\bin¶

Your build produces the llama.cpp executables, but the ROCm runtime DLLs and pre-compiled GPU kernel data are not copied automatically. You need to place them alongside your binaries.

Side note: you can probably just download one of the lemonadesdk llamacpp releases and copy the two rocblas and hipblaslt folders into your bin and be fine, though Ill show you what you need to copy if you don't want to do that.

Destination (all files go here):

C:\Code\llamacpp-rocm\llama.cpp\build\bin\

Option A: Copy specific files (minimum required)¶

From C:\Code\llamacpp-rocm\.venv\Lib\site-packages\_rocm_sdk_core\bin\:

From C:\Code\llamacpp-rocm\.venv\Lib\site-packages\_rocm_sdk_libraries\bin\:

Folders (copy the entire folder, not just the file):

After copying, your build\bin\ should contain:

build\bin\
├── amd_comgr.dll
├── amdhip64_7.dll
├── amdocl64.dll
├── hipblas.dll
├── hiprtc0716.dll
├── hiprtc-builtins0716.dll
├── libhipblaslt.dll
├── origami.dll
├── rocblas\
│   └── library\
│       └── *.hsaco, *.dat, *.co
├── rocblas.dll
├── rocm_kpack.dll
├── rocsolver.dll
├── hipblaslt\
│   └── library\
│       └── gfx1100\
│           └── *.co, *.dat.zlib, *.hsaco
├── llama-server.exe
├── llama-cli.exe
├── ... (other llama.cpp files)

Option B: Just copy everything (simpler, slightly larger)¶

Open two File Explorer windows:

• Source window 1: C:\Code\llamacpp-rocm\.venv\Lib\site-packages\_rocm_sdk_core\bin\ → Select all .dll files → Copy
• Source window 2: C:\Code\llamacpp-rocm\.venv\Lib\site-packages\_rocm_sdk_libraries\bin\ → Select all .dll files plus the rocblas\ folder and the hipblaslt\ folder → Copy
• Destination window: C:\Code\llamacpp-rocm\llama.cpp\build\bin\ → Paste everything

This grabs a few extra DLLs (MIOpen, hipfft, rocsparse, etc.) that llama.cpp doesn't strictly need, but they're harmless.

Verify¶

In your shell, from C:\Code\llamacpp-rocm\llama.cpp\build\bin\:

.\llama-server.exe --list-devices

If it prints your GPU info (e.g. HGGRT device 0: AMD Radeon RX 7900 XT, gfx1100) instead of crashing or throwing a DLL error, you're done.

Step 7: Run¶

Your binaries will be at build\bin\:

.\llama-server.exe -m C:\path\to\your\model.gguf --host 0.0.0.0 --port 8080 -ngl 99

Done!¶

Troubleshooting¶

"clang not found" or "CMake could not a working compiler"¶

Make sure you're in the Developer PowerShell and that $llvmBin is correct:

# Should print a full path to clang.exe
& "$llvmBin/clang.exe" --version

If rocm-sdk path --root returns empty, re-run rocm-sdk init.

"cannot find ROCm device library" / missing .bc files¶

Verify the bitcode path exists:

Test-Path "$rocmRoot/lib/llvm/amdgcn/bitcode"
# If False, find where the .bc files actually are:
Get-ChildItem -Recurse -Filter "oclc_abi_version_400.bc" $rocmRoot | Select-Object -First 3 -ExpandProperty DirectoryName

Then update -DCMAKE_CXX_FLAGS to point at the correct directory.

rocm-sdk path --bin doesn't contain clang¶

This is expected. The top-level bin/ has hipcc.exe and shims. The real compilers are in $rocmRoot/lib/llvm/bin/. Always use $llvmBin for the compiler flags.

CMake warning: "ROCM_HOME unused"¶

Harmless. The current llama.cpp CMake resolves hip/hipBLAS via CMAKE_PREFIX_PATH and doesn't read ROCM_HOME directly. You can drop it from the command.

CMake warning: "OpenSSL not found"¶

Disables HTTPS in the llama-server tool. Irrelevant for local use. If you need it, install OpenSSL for Windows and pass -DOPENSSL_ROOT_DIR=....

Comparing Results (Vulkan vs ROCm)¶

TLDR¶

ROCm with these set of patches is better, even if you take a hit a little bit on token gen. The model will feel more responsive, and prompt processing is way higher. Shout out to stew675 for the patches!

First Test (token generation)¶

Using llamacpp's llama-bench and the speed_bench.py python files, here are some initial results with ROCm + patches.

Heres how im starting llama-server for this test:

(llama-benchmarks) PS C:\Users\Troy\Desktop\llama-benchmarks> .\llama-server.exe -m "C:/Code/llamacpp/models/Qwen3.8-27B-IQ4_XS.gguf" -ngl 99 -c 80000 -fa on -np 1 -ctk q4_0 -ctv q4_0 -b 512 -ub 256 --spec-type draft-mtp,ngram-mod --spec-draft-n-max 3 --spec-ngram-mod-n-match 24 --spec-ngram-mod-n-min 8 --spec-ngram-mod-n-max 32 --n-predict 30000 --port 8080 --jinja

And here's the test command that was used.:

(llama-benchmarks) PS C:\Users\Troy\Desktop\llama-benchmarks> uv run python .\speed_bench.py `
>>   --url localhost:8080 `
>>   --bench qualitative `
>>   --category coding,reasoning `
>>   --osl 256 `
>>   --limit 5 `
>>   --concurrency 1 `
>>   --output vulkan.json

The only thing changing here is the llama-server.exe between runs (vulkan vs rocm) and of course the output file name.

Second Test¶

Arguably the more important test is prompt processing. Im using llama-benchy here to run my actual llama-server config loaded with my models.ini file and test against that.

; models.ini
[*]
host = 0.0.0.0
port = 11434
jinja = true
; log-verbose = 4
parallel = 1
flash-attn = true

[Qwen3.8-27B]
model = C:/Code/llamacpp/models/Qwen3.8-27B-IQ4_XS.gguf
n-gpu-layers = 99
ctx-size = 80000
flash-attn = on
parallel = 1
cache-type-k = q4_0
cache-type-v = q4_0
temp = 1.0
top-p = 0.95
top-k = 20
min-p = 0.0
presence-penalty = 0.0
repeat-penalty = 1.0
no-mmproj-offload = true
no-mmap = true
chat-template-kwargs = {"reasoning_effort": "xhigh"}
n-predict = 30000
batch-size = 512
ubatch-size = 256
load-mode = mlock
spec-type = draft-mtp,ngram-mod
spec-draft-n-max = 3
spec-ngram-mod-n-match = 24
spec-ngram-mod-n-min = 8
spec-ngram-mod-n-max = 32

# llama-benchy command
llama-benchy --base-url http://127.0.0.1:11434 --model "Qwen3.8-27B" --depth 0 4096 8192 16384 32768 65536 79000 --latency-mode generation --format json --save-result rocm.json

Prompt processing (tok/s) — higher is better

Token generation (tok/s) — higher is better

Time to first token (s) — lower is better