Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Simplified the MPS support based on your suggestion @t-vi (CC @Andrei-Aksionov).
It's a bit trickier to have unit tests for this behavior now (because we can't force to run the MPS code on non-Mac machines now), but it's fine because we know the previous unit tests for equivalency passed. And sure, we can have macOS tests for this alternative path, but the problem is that MPS has subtle numerical differences, so I don't expect the results of MPS and CPU to be equivalent. But we can address this is in #1725
Note that I also changed the default type for MPS to bf16 because it seems supported now. If we use regular float 16, we can get weird inference results for many models, which is why I changed it.