What to ask: rank a list of clarifying questions

[ilyalasy]

Build a model for ranking clarifying questions given an instruction.
See What to ask.

Questions to rank are in question_bank.csv, see dataset.

Models to try:

1. BM25 Ranker (no training needed) - used in official baseline
2. Semantic search (no training needed) - sentence-transformers
3. Polyencoder ranker - parlai
4. Dense Passage Retrieval - facebookresearch
5. ColBERT - stanford-futuredata

[Darth-Carrotpie]

Note: Some of the clarifying questions seem to be in broken English or contain spelling errors. Probably should keep that in mind when testing.

[ilyalasy]

FYI: I preprocessed data and splitted it into train/dev, so we should use these csvs for our models.
Notebook used for preprocessing is here.