Published May 12, 2024 | Version v1
Software Open

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving

Authors/Creators

  • 1. ROR icon Peking University
  • 2. ROR icon University of California, San Diego
  • 3. Independent Researcher

Files

DistServe.zip

Files (661.6 kB)

Name Size Download all
md5:804a0445e69b6d2f293e5c74e890acc9
661.6 kB Preview Download

Additional details

Software

Repository URL
https://github.com/LLMServe/DistServe/tree/main
Programming language
Python
Development Status
Active