Tips for those who want to get the last_hidden_state #723
Unanswered
MilkClouds
asked this question in
Q&A
Replies: 1 comment
-
@MilkClouds see also, #721 this is good topic for Discussions, but should probably formalize these into documentation... and make process a bit cleaner (ie support timm properly, although as in 721 you can call a timm specific method on trunk explicitly to get this) |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
tldr:
explain:
some model(except for vision model for timm and openai) have
output_tokens
argument. by editing config or simply setting attributemodel.output_tokens
to True, you can getlast_hidden_state
(except CLS token). By setting attn_pool and proj to None, we can obtain the CLS token. If you find any errors in this source code, please report me.Beta Was this translation helpful? Give feedback.
All reactions