request_success_total metric not shown after upgrade vllm 0.5.5 -> 0.6.2 and turn on num-scheduler-steps option #9526
Unanswered
yongho-chang
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello recently i upgraded vllm 0.5.5 -> 0.6.2 and turn on num-scheduler-steps option.
after work.
below metric is not shown.
request_success_total
e2e_request_latency_seconds_sum
request_generation_tokens_sum
request_prompt_tokens_sum
is it bug?
is seems because of num-scheduler-steps option
my vllm options are bellow.
- --tensor-parallel-size
- "4"
- --load-format
- auto
- --max-model-len
- "8192"
- --block-size
- "16"
- --gpu-memory-utilization
- "0.95"
- --num-scheduler-steps
- "8"
- --uvicorn-log-level
- warning
- --disable-log-requests
Beta Was this translation helpful? Give feedback.
All reactions