Question on ThreadPoolExecutor and GIL Impact #8079
Unanswered
satshiv7983
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi team, i have one doubt.
In the vllm codebase, we're using ThreadPoolExecutor for model batch processing, which allows control to return to the event loop for handling more new queries. With Python's GIL allowing only one thread to execute at a time, is the processing actually happening in parallel, or is it just context-switching between threads?
How does the GIL affect this setup, and is there true parallelism with ThreadPoolExecutor in this context?
Beta Was this translation helpful? Give feedback.
All reactions