Kalsa
Engines / stew675 / stew675/llama-cpp-rdna-boosts

stew675/llama-cpp-rdna-boosts

AMDForkMIT

Speeds up decoding in llama.cpp on AMD GPUs with RDNA patches and a speculative-decoding clamp. It is a fork of ggml-org/llama.cpp.

Benchmarks

  • On 2x Radeon AI PRO R9700 (gfx1201) with Flash-Next (IQ4_NL), it decodes at 168.3 tok/s, against 73.9 tok/s for spec clamp off (2.3×). Source

Listing

Listed: Passed all seven checks.

Security review

Not reviewed yet.

Install

To install it, follow the project's own instructions on GitHub. The engine runs with your user's permissions. macOS doesn't sandbox it, so install only engines you trust.

Something wrong or out of date? Write to help@kalsa.io, or ask us to remove this page.