A collection of (stochastic) gradient descent algorithms with a unified interface.
Provide a very flexible framework to experiment with algorithm design for optimization problems that rely on (stochastic) gradients. Issues considered:
- minibatches
- learning rates, fixed, adaptive, annealing
- preconditioning
- momentum
- averaging
- ...