LogProbs - Incorrect?

[MM-IR]

Hi, I am working on some multi-choice tasks in LM_evaluation_harness, where I find your results inconsistent with existing results.

• So I have some questions regarding your LogProbs specified in your SamplingParams.
• Is that correct as the log_softmax(logits)?

[zhuohan123]

It should be log_softmax(logits) as in the code below.

vllm/vllm/model_executor/layers/sampler.py

Lines 74 to 79 in bda41c7

	# We use float32 for probabilities and log probabilities.
	# Compute the probabilities.
	probs = torch.softmax(logits, dim=-1, dtype=torch.float)
	# Compute the log probabilities (before applying top-p and top-k).
	logprobs = torch.log(probs)

Convert this issue to a discussion for future questions.

[MM-IR]

Oh sorry, I have noticed that. Can you support us to access the whole sentence logits/logProbs by autoregressive LMs?

There are some multi-choice tasks, requiring us to manipulate the generated tokens, instead of letting these LMs generate the completion tokens themselves.

[RanchiZhao]

same problem. How to get ppl in vllm api?