Kalsa
Engines / neurall / neurall/llama.cpp

neurall/llama.cpp

NVIDIAForkMIT

Streams experts from SSD or RAM into GPU passes, which speeds up prompt processing in llama.cpp on NVIDIA GPUs. It is a fork of ggml-org/llama.cpp.

Benchmarks

  • On 2x RTX 3090 (48 GiB VRAM), Ryzen 7 3700X, 128 GB DDR4 with Qwen 3.8 Next (IQ4_XS), it decodes at 52.7 tok/s, against 32.6 tok/s for official stock binary (+62%), and it reads prompts at 1902 tok/s, against 213 tok/s for official stock binary (8.9×). Source

Listing

Listed: Passed all seven checks.

Security review

Not reviewed yet.

Install

To install it, follow the project's own instructions on GitHub. The engine runs with your user's permissions. macOS doesn't sandbox it, so install only engines you trust.

Something wrong or out of date? Write to help@kalsa.io, or ask us to remove this page.