Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Return result with best log prob when all temperature fallbacks failed #356

Merged
merged 7 commits into from
Jul 20, 2023
Merged

Return result with best log prob when all temperature fallbacks failed #356

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
c860004
Resolve Inference Selection Bug
hoonlight Jul 17, 2023
0e58b3f
Refactor for better readability
hoonlight Jul 17, 2023
c9e07ab
Merge branch 'guillaumekln:master' into return-best-result
hoonlight Jul 19, 2023
ca28641
Filter out results with compression_ratio
hoonlight Jul 19, 2023
30e0efe
Refactor to avoid variable repetition
hoonlight Jul 19, 2023
19c867a
Fix incorrect index and perform minor refactoring
hoonlight Jul 19, 2023
a578a18
Remove final_temperature variable
hoonlight Jul 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 28 additions & 17 deletions faster_whisper/transcribe.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -578,10 +578,9 @@ def generate_with_fallback(
tokenizer: Tokenizer,
options: TranscriptionOptions,
) -> Tuple[ctranslate2.models.WhisperGenerationResult, float, float, float]:
result = None
avg_logprob = None
final_temperature = None
compression_ratio = None
decode_result = None
all_results = []
below_cr_threshold_results = []

max_initial_timestamp_index = int(
round(options.max_initial_timestamp / self.time_precision)
Expand All @@ -601,7 +600,6 @@ def generate_with_fallback(
"patience": options.patience,
}

final_temperature = temperature
result = self.model.generate(
encoder_output,
[prompt],
Expand All @@ -625,20 +623,28 @@ def generate_with_fallback(
text = tokenizer.decode(tokens).strip()
compression_ratio = get_compression_ratio(text)

decode_result = (
result,
avg_logprob,
temperature,
compression_ratio,
)
all_results.append(decode_result)

needs_fallback = False

if (
options.compression_ratio_threshold is not None
and compression_ratio > options.compression_ratio_threshold
):
needs_fallback = True # too repetitive
if options.compression_ratio_threshold is not None:
if compression_ratio > options.compression_ratio_threshold:
needs_fallback = True # too repetitive

self.logger.debug(
"Compression ratio threshold is not met with temperature %.1f (%f > %f)",
temperature,
compression_ratio,
options.compression_ratio_threshold,
)
self.logger.debug(
"Compression ratio threshold is not met with temperature %.1f (%f > %f)",
temperature,
compression_ratio,
options.compression_ratio_threshold,
)
else:
below_cr_threshold_results.append(decode_result)

if (
options.log_prob_threshold is not None
Expand All @@ -661,8 +667,13 @@ def generate_with_fallback(

if not needs_fallback:
break
else:
# all failed, select the result with the highest average log probability
decode_result = max(
below_cr_threshold_results or all_results, key=lambda x: x[1]
)

return result, avg_logprob, final_temperature, compression_ratio
return decode_result

def get_prompt(
self,
Expand Down