-
Notifications
You must be signed in to change notification settings - Fork 120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
update document to include DML isntruction #996
base: main
Are you sure you want to change the base?
Conversation
Co-authored-by: kunal-vaishnavi <[email protected]>
ce5379e
to
4c2d0e6
Compare
``` | ||
- `pillow` | ||
- `requests` | ||
- `torch` | ||
- Install torch with CUDA by following the [instructions](https://pytorch.org/get-started/locally/) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Install torch with CUDA by following the [instructions](https://pytorch.org/get-started/locally/) | |
- Please install torch by following the [instructions](https://pytorch.org/get-started/locally/). For getting ONNX models that can run on CUDA or DirectML, please install torch with CUDA and ensure the CUDA version you choose in the instructions is the one you have installed. |
## 3. Build `genai_config.json` and `processor_config.json` | ||
|
||
Currently, both JSON files needed to run with ONNX Runtime GenAI are created by hand. Because the fields have been hand-crafted, it is recommended that you copy the already-uploaded JSON files and modify the fields as needed for your fine-tuned Phi-3 vision model. [Here](https://huggingface.co/microsoft/Phi-3-vision-128k-instruct-onnx-cpu/blob/main/cpu-int4-rtn-block-32-acc-level-4/genai_config.json) is an example for `genai_config.json` and [here](https://huggingface.co/microsoft/Phi-3-vision-128k-instruct-onnx-cpu/blob/main/cpu-int4-rtn-block-32-acc-level-4/processor_config.json) is an example for `processor_config.json`. | ||
|
||
### For DirectML |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Replace
"provider_options": []
in genai_config.json
with
"provider_options": [
{
"dml" : {}
}
]
] | ||
``` | ||
|
||
### For CUDA |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Replace
"provider_options": []
in genai_config.json
with
"provider_options": [
{
"cuda" : {}
}
]
|
||
### CUDA | ||
```bash | ||
cd .. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is cd ..
needed?
Updated documents to include DirectML instructions for phi-3, phi-v, and awq-quantization