Skip to content

Have you compared with FasterTransformer #264

Answered by zhuohan123
linbojin asked this question in Q&A
Discussion options

You must be logged in to vote

Thanks for the question. Yes we have compared the performance with FasterTransformer in our research paper (will be released soon). We can achieve up to a up to 22x speedup compared to FasterTransformer. The main gain comes from the PagedAttention and continuous batching implemented in vLLM.

Replies: 1 comment 5 replies

Comment options

You must be logged in to vote
5 replies
@deepindeed2022
Comment options

@void-main
Comment options

@zhuohan123
Comment options

@julian-q
Comment options

@TheAthleticCoder
Comment options

Answer selected by zhuohan123
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
6 participants
Converted from issue

This discussion was converted from issue #256 on June 26, 2023 18:26.