Skip to content

tokens个数怎么计算的？ #595

Closed Answered by zRzRzRzRzRzRzR
chenmonster asked this question in Q&A
`tokens`个数怎么计算的？ #595
@chenmonster chenmonster
Dec 14, 2023 · 1 comment
Answered by zRzRzRzRzRzRzR Return to top
Discussion options

chenmonster
Dec 14, 2023

You must be logged in to vote
Answered by zRzRzRzRzRzRzR Dec 14, 2023

用hf tokenizer

View full answer

Replies: 1 comment

Comment options

zRzRzRzRzRzRzR
Dec 14, 2023
Maintainer

You must be logged in to vote
0 replies
Answer selected by zRzRzRzRzRzRzR
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants
@chenmonster @zRzRzRzRzRzRzR