Skip to content

Commit

Permalink
Disable triton profile tests due to genai-perf bug in 24.09 - should …
Browse files Browse the repository at this point in the history
…be restored in 24.10
  • Loading branch information
rmccorm4 committed Oct 25, 2024
1 parent 960bd76 commit febed9b
Showing 1 changed file with 9 additions and 0 deletions.
9 changes: 9 additions & 0 deletions tests/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,15 @@ def _infer(model, prompt=None, protocol=None):
run(args)

def _profile(model, backend):
# FIXME: WAR for genai-perf bug in 24.09, remove in 24.10
import genai_perf

if genai_perf.__version__ == "0.0.6dev":
print(
"[WARNING] Skipping call to 'triton profile' due to known issue in genai-perf"
)
return

args = ["profile", "-m", model, "--backend", backend]
# NOTE: With default parameters, genai-perf may take upwards of 1m30s or 2m to run,
# so limit the genai-perf run with --request-count to reduce time for testing purposes.
Expand Down

0 comments on commit febed9b

Please sign in to comment.