Using Levenshtein C module instead of custom cpp code

[BramVanroy]

The following line

CharacTER/CharacTER.py

Line 54 in c4b25cb

result = self.ed_wrapper.wrapper(hyp_c, ref_c, len(hyp_c), len(ref_c), norm)

can be replaced by

Levenshtein.distance(hyp_c, ref_c) / norm

If the C extension module is installed (Levenshtein) this computation will be blazing fast and compatible with different platforms. Currently, how CharacTER is written with a precompiled SO, this is not the case.

Making the change suggested above will lead to better compatibility with close to zero performance loss due to the C implementation in Levenshtein. In fact, the whole EditDistance class seems unnecessary as pre_score can be replaced by a single line:

pre_score = Levenshtein.distance(hyp_words, ref_words) / len(ref_words)

I tested this with a large corpus and results were always the same as the current implementation.