Speculative decoding for high-throughput long-context inference
What's this blog post about?
Company
Together AI
Date published
Sept. 5, 2024
Author(s)
Jian Chen, Vashisth Tiwari, Ranajoy Sadhukhan, Yunho Jin, Zhuoming Chen, Jinyuan Shi, Ian En-Hsu Yen, Avner May, Beidi Chen
Word count
2002
Hacker News points
None found.
Language
English