Skip to content

Latest commit

 

History

History
10 lines (7 loc) · 585 Bytes

minibatch-bootstrapped-gradients.md

File metadata and controls

10 lines (7 loc) · 585 Bytes

minibatch bootstrapped gradients




sort of like a data augmentation, but more like a gradient regularization. for some large batch, accumulate gradients over bootstrap resamples from that minibatch. for n resamples, divide learning rate by n (?)... or not? or multiply? i dunno. to do: investigate hyperparameter scaling in this regime. sheesh.