Kalsa
Engines / EricLBuehler / mistral.rs

mistral.rs

Apple M1–M5NVIDIAEngineMIT

Runs models in Rust with ISQ and UQFF quantization on Apple Silicon and NVIDIA GPUs.

Benchmarks

  • On v0.8.2 CUDA, GB10 (DGX Spark desk-side) with Q8 (Q8), it decodes at 46.8 tok/s, about the same speed as llama.cpp (46.4 tok/s), and it reads prompts at 2947 tok/s, against 2178.5 tok/s for llama.cpp (+35%). Source Note: The decode pair is about the same speed; the gain is on prefill

Listing

Listed: Passed all seven checks.

Security review

Not reviewed yet.

Install

To install it, follow the project's own instructions on GitHub. The engine runs with your user's permissions. macOS doesn't sandbox it, so install only engines you trust.

Something wrong or out of date? Write to help@kalsa.io, or ask us to remove this page.