推理的最小显存估计是多少？ #984
推理的最小显存估计是多少？
#984
-
|
我使用hf的估算工具，FP16推理所需最小显存是5.81G （ https://huggingface.co/spaces/hf-accelerate/model-memory-usage ） 官方Readme建议是13G 这个中间的差距，主要是什么占用呢？ |
Beta Was this translation helpful? Give feedback.
Answered by
zRzRzRzRzRzRzR
Mar 15, 2024
Replies: 1 comment
-
|
13G，5G是int4，另外你那个链接我也打不开 |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
foobra
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13G，5G是int4，另外你那个链接我也打不开