Vllm

🚀 vLLM Project Documentation

vLLM is an open - source tool for efficiently running and deploying large language models (LLMs). It aims to make running large - scale language models in a production environment easier and more cost - effective by optimizing memory management and computing resource utilization.

🚀 Quick Start

Visit our official documentation for more information:

• Installation Guide: Link
• Quick Start: Link
• Supported Models: Link

✨ Features

• Efficient Memory Management: Utilizes the PagedAttention mechanism to significantly reduce GPU memory usage.
• Support for Multiple Model Architectures: Covers mainstream open - source models such as GPT, BLOOM, and Mistral.
• Easy Deployment and Scalability: Offers a simple interface for easy integration into various applications.

📦 Installation

You can install vLLM via pip or from the source code:

pip install vllm

🤝 Project Contribution

We welcome any contributions and collaborations. Please refer to CONTRIBUTING.md to learn how to get involved.

📚 Citation

If you use vLLM in your research, please cite our paper:

@inproceedings{kwon2023efficient,
  title={Efficient Memory Management for Large Language Model Serving with PagedAttention},
  author={Woosuk Kwon and Zhuohan Li and Siyuan Zhuang and Ying Sheng and Lianmin Zheng and Cody Hao Yu and Joseph E. Gonzalez and Hao Zhang and Ion Stoica},
  booktitle={Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles},
  year={2023}
}

📦 Model Support

vLLM currently supports the following language model architectures:

• GPT Series: Including gpt2, gpt2 - xl, etc.
• BLOOM Series: Such as bigscience/bloom, etc.