Engines / LaurentZuijdwijk / llama.cpp adaptive spec decoding + Strix Halo Vulkan

llama.cpp adaptive spec decoding + Strix Halo Vulkan
Speeds up prompt reading in llama.cpp on AMD Strix Halo, using adaptive speculative decoding. It is a fork of ggml-org/llama.cpp.
Benchmarks
- On Radeon 8060S (Strix Halo) with Ornith-1.5-35B-A3B (Q4_K_M), it reads prompts at 1648.5 tok/s, against 870.5 tok/s for llama.cpp (+89%). Source
Listing
Listed: Passed all seven checks.
Security review
Not reviewed yet.
Install
To install it, follow the project's own instructions on GitHub. The engine runs with your user's permissions. macOS doesn't sandbox it, so install only engines you trust.
Something wrong or out of date? Write to help@kalsa.io, or ask us to remove this page.