Add script for n_best parameter in topp/topk

[PC91]

Implementation of this feature:

• Sort the score of all hypotheses when there are more than 1.
• Choose n_best results
• Update unittest

[vince62s]

Looks good thank you, have you tried to use it with this: https://github.com/OpenNMT/OpenNMT-py/blob/master/tools/mbr_bleu.py ?

[PC91]

Looks good thank you, have you tried to use it with this: https://github.com/OpenNMT/OpenNMT-py/blob/master/tools/mbr_bleu.py ?

I did not see this tool. Will try it with the new feature

[PC91]

Looks good thank you, have you tried to use it with this: https://github.com/OpenNMT/OpenNMT-py/blob/master/tools/mbr_bleu.py ?

@vince62s I tested on some configurations and that works well. Could you tell me how to test with this tool ? I understand that it outputs the hypothesis that has the maximum average BLEU score when comparing to others. But I'm not sure how it is related to the goodness of each hypothesis generated from topp/topk sampling. Thank you.

[vince62s]

You can read this paper: https://arxiv.org/pdf/2111.09388.pdf
also in tools there are two others script oracle_bleu and oracle_comet
based on oracle_comet and mbr_bleu you could add mbr_comet (as in the paper).
The principle is simple just compare all hypothesis between each other and select the one giving the best bleu (resp Comet) vs all others. It should give some improvement over beam search best score.