Replies: 1 comment 1 reply
-
|
你可以直接 uvicorn命令行运行 这样可以开启多个wokers |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
已解决，感谢大佬 |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
当workers大于1时，显示报错是模型未定义，即便将model设为全局变量也是一样。
所以想请问一下如何实现多进程呢？
还想和大家讨论一下如何在多进程调用一个llm的条件下如何节约显存的问题。
Beta Was this translation helpful? Give feedback.
All reactions