Kalsa
Engines / LaurentZuijdwijk / llama.cpp adaptive spec decoding + Strix Halo Vulkan

llama.cpp adaptive spec decoding + Strix Halo Vulkan

AMDForkMIT

Speeds up prompt reading in llama.cpp on AMD Strix Halo, using adaptive speculative decoding. It is a fork of ggml-org/llama.cpp.

Benchmarks

  • On Radeon 8060S (Strix Halo) with Ornith-1.5-35B-A3B (Q4_K_M), it reads prompts at 1648.5 tok/s, against 870.5 tok/s for llama.cpp (+89%). Source

Listing

Listed: Passed all seven checks.

Security review

Not reviewed yet.

Install

To install it, follow the project's own instructions on GitHub. The engine runs with your user's permissions. macOS doesn't sandbox it, so install only engines you trust.

Something wrong or out of date? Write to help@kalsa.io, or ask us to remove this page.