AI and Agents

LLM Inference and Serving

Listed in editorial order. Click a column to re-sort the whole list.

Press / to search. Tap a tag to filter. Click any row for details.

Search and filter

Results

Row number Tags
A high-performance serving framework for large language models and multimodal models.
sgl-project/github.com/sgl-project/sglang / /2,758,431 downloads/month
A high-throughput and memory-efficient inference and serving engine for LLMs.
vllm-project/github.com/vllm-project/vllm / /2,110,921 downloads/month
Run and fine-tune large language models on Apple Silicon with MLX.
ml-explore/github.com/ml-explore/mlx-lm / /547,306 downloads/month

Know a project that belongs here?

Tell us what it does and why it stands out.