Kalsa
Engines / jundot / oMLX

oMLX

Apple M1–M5EngineApache-2.0

Speeds up prompt reading on Apple Silicon by pairing a second Mac.

Benchmarks

  • On M3 Ultra (512 GB) with an M5 Max 128 GB over Thunderbolt 5 with DeepSeek V4.1 Flash (oQ3e), it reads prompts at 3000 tok/s, against 1060 tok/s for This Mac alone (2.8×). Source Note: A 64K-token prompt; the author reports 3,000-3,160 tok/s with the second Mac, and the lower end is used here. The feature is experimental. It needs a second Mac on the network with Remote Login turned on in System Settings, the same oMLX version on both Macs and Distributed Inference enabled in settings. The model is not split: decoding and the cache stay on the first Mac. This run used INT8 Activation Prefill on the helper's layers, which changes numerics; the author reports greedy output matched on the tested prompts.

Listing

Listed: Passed all seven checks.

Security review

Not reviewed yet.

Install

To install it, follow the project's own instructions on GitHub. The engine runs with your user's permissions. macOS doesn't sandbox it, so install only engines you trust.

Something wrong or out of date? Write to help@kalsa.io, or ask us to remove this page.