The Economics of Speculative Decoding
Two underexplored axes: what MoE routing does to the decode roofline, and how
compressed attention takes away the slack that used to make speculated tokens free.
compressed attention takes away the slack that used to make speculated tokens free.
Summary not yet generated for this article.
Comments (3 of ) ⌛