Kalsa
Engines / ewhacc / tandem.cpp

tandem.cpp

NVIDIAEngineMIT

Stores MoE experts in RAM tiers for models running on NVIDIA GPUs.

Benchmarks

  • On RTX 5090 + 128 GB RAM (PC) with Maya-XS GGUF (routed experts) (IQ1_S/IQ2_XXS, 1.63 bpw), it decodes at 60.3 tok/s, against 39 tok/s for Maya (+55%). Source Note: Against Maya's DSpark arm at 54.9 tok/s the gain is +9.8%

Listing

Listed: Passed all seven checks.

Security review

Not reviewed yet.

Install

To install it, follow the project's own instructions on GitHub. The engine runs with your user's permissions. macOS doesn't sandbox it, so install only engines you trust.

Something wrong or out of date? Write to help@kalsa.io, or ask us to remove this page.