Pinned Loading

1. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 21.8k 3.1k

2. bytedance/lightseq bytedance/lightseq Public

   LightSeq: A High Performance Library for Sequence Processing and Generation

   C++ 3.1k 325

3. microsoft/Megatron-DeepSpeed microsoft/Megatron-DeepSpeed Public

   Forked from NVIDIA/Megatron-LM

   Ongoing research training transformer language models at scale, including: BERT & GPT-2

   Python 1.7k 328

4. AniZpZ/AutoSmoothQuant AniZpZ/AutoSmoothQuant Public

   An easy-to-use package for implementing SmoothQuant for LLMs

   Python 66 4

5. zhengjian2322/net2net zhengjian2322/net2net Public

   Python 4 1

6. QQQ QQQ Public

   QQQ is an innovative and hardware-optimized W4A8 quantization solution.

   Python 22