Pinned Loading

1. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 95.9k 24.1k

2. deepspeedai/DeepSpeed deepspeedai/DeepSpeed Public

   DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

   Python 43.3k 5.1k

3. uccl-project/uccl uccl-project/uccl Public

   UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

   C++ 1.6k 187

4. InferMatrixLab/vllm-rlt InferMatrixLab/vllm-rlt Public

   A vLLM-inspired inference engine for looped transformers, with loop-level scheduling and execution across composable stages

   Python 211 34