Kalsa
Hardware / NVIDIA

Engines for NVIDIA

16 engines run on NVIDIA.

architectds/Strataby architectds

Speeds up prompt processing in Strata on NVIDIA GPUs.

Fork of Niko1221/Strata
NVIDIAForkMIT
Faster prefill
+94% faster prompt reading on NVIDIA GPU
DwarfStar 4by antirez

Runs large models such as DeepSeek V4 Flash and GLM 5.3 Flash on Apple Silicon, NVIDIA and AMD.

Apple M1–M6NVIDIAAMDEngineMIT
Bigger modelsMultiple machinesNew hardware
No benchmarks yet
GenerelSchwerz/llama.cppby GenerelSchwerz

Changes how llama.cpp places its cache on NVIDIA GPUs, and decodes Qwen3.6 35B faster than stock.

Fork of ggml-org/llama.cpp
NVIDIAForkMIT
Faster decode
2.3× faster decoding than llama.cpp on NVIDIA GPU
Lidenburg/llama.cppby Lidenburg

Adds an expert cache with GPU and RAM tiers to llama.cpp on NVIDIA GPUs.

Fork of ggml-org/llama.cpp
NVIDIAForkMIT
Faster decode
+60% faster decoding than without it on NVIDIA GPU
llama.cppby ggml-org

Serves as the reference upstream and runs on Apple Silicon, NVIDIA, AMD, Intel and CPU.

Apple M1–M5NVIDIAAMDIntelCPUReferenceMIT
The reference engine
llama.cpp adaptive KV streamingby RaymondHuang210129

Streams the KV cache between host RAM and VRAM in a llama.cpp fork for NVIDIA GPUs.

Fork of ggml-org/llama.cpp
NVIDIAForkMIT
Faster decode
10.4× faster decoding on NVIDIA GPU
llama.cpp MoE expert cacheby thecodacus

Keeps MoE experts in a VRAM-resident cache in a llama.cpp fork for NVIDIA GPUs.

Fork of ggml-org/llama.cpp
NVIDIAForkMIT
Faster decodeFaster prefill
+21% faster decoding than without it on NVIDIA GPU
mistral.rsby EricLBuehler

Runs models in Rust with ISQ and UQFF quantization on Apple Silicon and NVIDIA GPUs.

Apple M1–M5NVIDIAEngineMIT
Faster prefill
+35% faster prompt reading than llama.cpp on NVIDIA GPU
neurall/llama.cppby neurall

Streams experts from SSD or RAM into GPU passes, which speeds up prompt processing in llama.cpp on NVIDIA GPUs.

Fork of ggml-org/llama.cpp
NVIDIAForkMIT
Faster decodeFaster prefill
+62% faster decoding on NVIDIA GPU
neutronstarby giannisanni

Adds batch prefill for GLM models to ds4 on NVIDIA GPUs.

Fork of antirez/ds4
NVIDIAForkMIT
Faster prefill
Not recommended: ~0.4 tok/s generation (about 2.5 s per token; the README's own interactive figure is ~2.1 tok/s)21.7× faster prompt reading than without it on NVIDIA GPU
Ollamaby ollama

Serves as the reference upstream: a UX wrapper with a model library.

Apple M1–M5NVIDIAAMDCPUReferenceMIT
The reference engine
spiritbuun/buun-llama-cppby spiritbuun

Adds a trellis-coded 3-bit KV cache codec to llama.cpp for NVIDIA and AMD GPUs.

Fork of ggml-org/llama.cpp
NVIDIAAMDForkMIT
Quality
No benchmarks yet
Strataby Niko1221

Serves as the reference upstream for Strata forks on NVIDIA, AMD and Apple Silicon.

Apple M1–M6NVIDIAAMDReferenceMIT
The reference engine
tandem.cppby ewhacc

Stores MoE experts in RAM tiers for models running on NVIDIA GPUs.

NVIDIAEngineMIT
Faster decode
+55% faster decoding on NVIDIA GPU
turbo-tan/llama.cpp-tq3by turbo-tan

Adds TQ3 weight formats to llama.cpp and an FP4 prompt path on Blackwell GPUs.

Fork of ggml-org/llama.cpp
NVIDIAForkMIT
Faster prefill
2.6× faster prompt reading than without it on NVIDIA GPU
vLLMby vllm-project

Serves as the reference upstream for NVIDIA, AMD and Intel GPUs and CPU.

NVIDIAAMDIntelCPUReferenceApache-2.0
The reference engine