add ct2 server parameters

[900groove]

fix: #2001

This pull request will allow to set the server specs for CTranslate2

Experiment

Setting

server_conf

{
    "models_root": "./available_models",
    "models": [
        {
            "id": 100,
            "ct2_model": "ende_ctranslate2/",
            "model": "ende_ctranslate2/",
            "timeout": 600,
            "device": "cpu",
            "on_timeout": "to_cpu",
            "load": true,
            "tokenizer": {
                "type": "sentencepiece",
                "model": "sentencepiece.model"
            },
            "inter_threads": 1,
            "intra_threads": 1,
            "compute_type": "default",
            "opt": {
                "beam_size": 5
            }
        }
    ]
}

Data

• The model uses the CTranslate2 sample
• Translation is the average translation time of the top 100 sentences from toy-ende.

Performance

Process evaluation by changing inter_threads and intra_threads

inter_threads	intra_threads	time_avg[sec]
1	1	0.55
1	8	0.29
8	1	0.57
8	8	1.82

[francoishernandez]

Hi there,
Sorry this went a bit under my radar.
Thanks for you contribution, but wouldn't it be better/cleaner to pass some ct2_kwargs dict as kwargs of the CT2 Translator constructor to allow all possible parameters in a transparent way, rather than manually adding each of them in the code?

[900groove]

@francoishernandez
Thank you for your comment!
I'm not entirely sure I understand your comment. Is setting parameters in a way other than kwargs intended to look like environment variables?

[francoishernandez]

I mean that we could do something similar to what is done for pyonmttok, where all the tokenizer kwargs are in a "tokenizer" key in the config, and then passed automatically to the constructor:

OpenNMT-py/onmt/translate/translation_server.py

Lines 691 to 697 in 7e8c7c2

	tokenizer_params = dict(tokenizer_opt["params"])
	for key, value in tokenizer_opt["params"].items():
	if key.endswith("path"):
	tokenizer_params[key] = os.path.join(
	self.model_root, value)
	tokenizer = pyonmttok.Tokenizer(mode,
	**tokenizer_params)

[funboarder13920]

I implemented the ct2 params in #2104 using two conf dictionaries to allow all ct2 params to be forwarded.