Use initializer for soft-sort by default?

[marcocuturi]

@JTT94 , shouldn't we consider adding the sorting initializer by default to soft-sort?

[JTT94]

This makes sense to me. Perhaps the gaussian initialiser also makes sense as a default. I don't think it ever "hurts", except in perhaps maybe in dimension over 50

[marcocuturi]

I have seen in some experiments the Gaussian initializer perform slightly worse. But I cannot imagine a case where the 1D initializer does not work, specially for small epsilon. I really believe this should be the default approach...

[JTT94]

That is a shame about the Gaussian initializer. From the experiments I did and from Brandon's it seemed quite consistent.

Regarding the soft sort. What is the best way to implement this? I imagine fixing the default initializer for the soft sort specific 'solve' function in "tools" would be easiest.

[gjhuizing]

Tagging along on this issue since I noticed the sorting initializer is the only one that is not reverse-mode differentiable because of the while loop.

Locally I added a small force_scan argument to fix this, but maybe you'd want to wrap this in an ott.math.fixed_point_loop?

[JTT94]

The idea behind using initialisers and implicit differentiation is that the initialisers do not need to be differentiable.