-
Notifications
You must be signed in to change notification settings - Fork 80
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use initializer for soft-sort by default? #275
Comments
This makes sense to me. Perhaps the gaussian initialiser also makes sense as a default. I don't think it ever "hurts", except in perhaps maybe in dimension over 50 |
I have seen in some experiments the Gaussian initializer perform slightly worse. But I cannot imagine a case where the 1D initializer does not work, specially for small |
That is a shame about the Gaussian initializer. From the experiments I did and from Brandon's it seemed quite consistent. Regarding the soft sort. What is the best way to implement this? I imagine fixing the default initializer for the soft sort specific 'solve' function in "tools" would be easiest. |
Tagging along on this issue since I noticed the sorting initializer is the only one that is not reverse-mode differentiable because of the Locally I added a small |
The idea behind using initialisers and implicit differentiation is that the initialisers do not need to be differentiable. |
@JTT94 , shouldn't we consider adding the sorting initializer by default to soft-sort?
The text was updated successfully, but these errors were encountered: