adding auto langdetect and cleaning up keywords extracted by yake

[ahmednasserswe]

Description

applying the following to improve yake keywords extraction

1- Replace ’ with ' and similar for other characters

2- Keep the longest keyword of if there is an overlap between two keywords. (“good” and “good morning” keep “good morning”)

3- If language is not specified or is “auto” perform language detection with CLD

Reference: CV2-4909 (to provide additional context)

How has this been tested?

locally + already existing tests

[DGaffney]

Please write unit tests for keep_largest_overlapped_keywords.

[computermacgyver]

@ahmednasserswe Per the Jira ticket, let's use CLD - https://pypi.org/project/gcld3/ , which is a newer/better library than langdetect

[skyemeedan]

I think this would be easier to implement if there are tests added that cover the substitutions. And this will save lots of time later as we add more cases to substitute

At a higher level, do we know if the "sub keyword" examples we are seeing are coming from the same text (extracting both "Biden" and "Joe Biden") which hopefully will be well handled by this approach vs multiple texts (one article referring to "Biden" and another to "President Biden") which might need some other post-processing