Support for Custom trained llm models from hugging face #2392
Unanswered
shrirudragoud
asked this question in
Q&A
Replies: 1 comment 1 reply
-
|
If the architecture of your model remains unchanged during training, it is supported in vLLM. You can check out all the supported models at this page. |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
I've encountered the same issue. Generally, when we fine-tune a model based on what Hugging Face provides and save it, a file in a format that can be loaded by Hugging Face is generated. How can I use such a saved model in vLLM? |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I am Training my own model using the hugging face mistral llm and i want to know how can i use the vllm for my own trained model which i can run on my own onprem server
Beta Was this translation helpful? Give feedback.
All reactions