-
Notifications
You must be signed in to change notification settings - Fork 120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
LoRA support in model builder #955
Conversation
8f39891
to
1089d43
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please update the README with any new options and usage of this feature? https://github.com/microsoft/onnxruntime-genai/blob/main/src/python/py/models/README.md
65bba01
to
a36a98a
Compare
f665ef4
to
d43f4c5
Compare
d43f4c5
to
68e85cb
Compare
68e85cb
to
2687e60
Compare
Can you add onnxruntime-genai/src/python/py/models/builder.py Lines 3181 to 3213 in 2687e60
|
4333766
to
2854ff2
Compare
This can be done in another PR but we should add a LoRA model such as this one in the CIs. onnxruntime-genai/test/python/_test_utils.py Lines 55 to 77 in 0f59a90
The models in
|
This PR adds LoRA MatMul changes in model builder. It includes changes made by Kunal and few changes done to make it work with Olive.
It covers the scenario where base_layer of float and adapters are float.
This PR will be followed with support for quant models scenario