Kalsa
Engines / jjang-ai / vMLX

vMLX

Apple M1–M5EngineApache-2.0

Cuts memory use on Apple Silicon by loading only some of each MoE model's experts.

What it does better

Loading half of the experts in each layer (smelt 50) cuts active RAM for Nemotron-Cascade-2-30B-A3B-JANG_4M from 17,408 MB to 9,529 MB, a 45% saving, on an M3 Ultra with 128 GB

Trade-off: decode falls from 89.9 to 66.5 tok/s because expert swaps hit the SSD on the hot path, and the author published no quality measurement, only that the output stayed coherent. It needs an MoE model in JANG format and turns vision off.

Source

Benchmarks

The author hasn't published benchmarks yet.

Listing

Listed: Passed all seven checks.

Security review

Not reviewed yet.

Install

To install it, follow the project's own instructions on GitHub. The engine runs with your user's permissions. macOS doesn't sandbox it, so install only engines you trust.

Something wrong or out of date? Write to help@kalsa.io, or ask us to remove this page.