Kalsa

Locals

Apple M2EngineApache-2.0

Serves many requests at the same time on Apple M2 with higher total throughput.

What it does better

271 tok/s aggregate decode across 8 concurrent requests, against 87 tok/s for mlx-lm; total throughput, not the speed of one stream

Source

Benchmarks

The author hasn't published benchmarks yet.

Listing

Listed: Passed all seven checks.

Security review

Not reviewed yet.

Install

To install it, follow the project's own instructions on GitHub. The engine runs with your user's permissions. macOS doesn't sandbox it, so install only engines you trust.

Something wrong or out of date? Write to help@kalsa.io, or ask us to remove this page.