AI and Agents

LLM Serving

Listed in editorial order. Click a column header to re-sort the whole list.

Press / to search. Tags open their category. Click a row for details.

Search and filter

Results

Row number Tags
A high-throughput and memory-efficient inference and serving engine for LLMs.
vllm-project/github.com/vllm-project/vllm /95,485 stars /
A high-performance serving framework for large language models and multimodal models.
sgl-project/github.com/sgl-project/sglang /38,157 stars /
DeepSeek's production inference engine for DeepSeek models, with MoE expert parallelism.
deepseek-ai/github.com/deepseek-ai/DeepInfer /22,385 stars /

Know a project that belongs here?

Tell us what it does and why it stands out.