Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Agile tag is a way for MLPerf to stay agile & make early bets on upcoming hot benchmarks.
It allows adopting viral benchmarks while also providing a way to update, refresh or tweak them as the ML landscape changes quickly instead of getting locked-in to a 2 year cadence.
Ideally all benchmarks should have the agility to refresh if landscape warrants faster change e.g. update the sequence length of existing LLM models or tweak outdated architectures but we should also balance the churn to reduce submitter burden and prolong investment !/$.