nano-vllm

github.com ↗

- nano-vllm is a lightweight vLLM implementation built from scratch, providing fast offline inference with optimization techniques such as prefix caching, tensor parallelism, and CUDA graph.