Does this tool work with quanitzed models? #482
Unanswered
bartowski1182
asked this question in
Q&A
Replies: 4 comments
-
|
That's exactly what I was looking into. Specifically GPTQ models like: https://huggingface.co/TheBloke/based-30B-GPTQ |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
I also wondering how to start the quantize work. Looking forward to some helps. |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
looking forward to update |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Looking forward to support quantized models. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Haven't found any reference so I assume no, but wanted to check. would be really cool but also presumably insane levels of work
Beta Was this translation helpful? Give feedback.
All reactions