Kalsa
Engines / thecodacus / llama.cpp MoE expert cache

llama.cpp MoE expert cache

NVIDIAForkMIT

Keeps MoE experts in a VRAM-resident cache in a llama.cpp fork for NVIDIA GPUs. It is a fork of ggml-org/llama.cpp.

Benchmarks

  • On RTX 3060 12GB with Qwen3.6-35B-A3B (Q4_K_M), it decodes at 51.3 tok/s, against 42.3 tok/s for expert cache off (+21%), and it reads prompts at 1880 tok/s, against 1143 tok/s for expert cache off (+64%). Source

Listing

Listed: Passed all seven checks.

Security review

Not reviewed yet.

Install

To install it, follow the project's own instructions on GitHub. The engine runs with your user's permissions. macOS doesn't sandbox it, so install only engines you trust.

Something wrong or out of date? Write to help@kalsa.io, or ask us to remove this page.