-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error when trying to run with a Quantized base model #138
Comments
The error you're encountering, ValueError: Target modules ['q_proj', 'k_proj', 'v_proj'] not found in the base model, suggests that the PEFT configuration is looking for certain layers in the model that are named differently or do not exist. Given your model structure, it looks like the quantized model has replaced the standard projection layers (q_proj, k_proj, v_proj) with qkv_proj in FusedLlamaAttentionForQuantizedModel. possible solution:-
example snippet:-
hope this helps, |
Hello. I have been trying to run the multi task llama7b models with Bloke's llama 7b GPTQ(https://huggingface.co/TheBloke/Llama-2-7B-GPTQ) as the base.
While running the same in google colab, I get the error when trying to load PEFT from pre trained:
After a bit of searching, It says I'll have to re train the PEFT model by using a different config. Is there anything I can do? (other than training)
For debugging purposes, value of 'model' before PEFT is used:
The text was updated successfully, but these errors were encountered: